Introduction
Building on our foundation from Part 2, we'll now implement advanced AWS services and patterns used in production environments. This includes load balancers, auto-scaling, container orchestration, caching, and content delivery networks.
Architecture Overview
We'll enhance our infrastructure with:
- Application Load Balancer (ALB) for traffic distribution
- Auto Scaling Groups for automatic capacity management
- ECS Fargate for containerized applications
- ElastiCache for high-performance caching
- CloudFront CDN for global content delivery
- Secrets Manager for secure credential storage
Application Load Balancer (ALB)
ALB operates at Layer 7 and provides advanced routing capabilities.
ALB Configuration
# Application Load Balancer
resource "aws_lb" "main" {
name = "${var.environment}-alb"
internal = false
load_balancer_type = "application"
security_groups = [aws_security_group.alb.id]
subnets = aws_subnet.public[*].id
enable_deletion_protection = var.environment == "prod" ? true : false
# Access logs (optional but recommended)
access_logs {
bucket = aws_s3_bucket.alb_logs.id
prefix = "alb"
enabled = true
}
tags = {
Name = "${var.environment}-alb"
Environment = var.environment
}
}
# Target groups for different services
resource "aws_lb_target_group" "web" {
name = "${var.environment}-web-tg"
port = 80
protocol = "HTTP"
vpc_id = aws_vpc.main.id
health_check {
enabled = true
healthy_threshold = 2
interval = 30
matcher = "200"
path = "/health"
port = "traffic-port"
protocol = "HTTP"
timeout = 5
unhealthy_threshold = 2
}
# Stickiness for session-based applications
stickiness {
type = "lb_cookie"
cookie_duration = 86400
enabled = true
}
tags = {
Name = "${var.environment}-web-tg"
}
}
resource "aws_lb_target_group" "api" {
name = "${var.environment}-api-tg"
port = 8080
protocol = "HTTP"
vpc_id = aws_vpc.main.id
health_check {
enabled = true
healthy_threshold = 2
interval = 30
matcher = "200"
path = "/api/health"
port = "traffic-port"
protocol = "HTTP"
timeout = 5
unhealthy_threshold = 2
}
tags = {
Name = "${var.environment}-api-tg"
}
}
# ALB Listeners
resource "aws_lb_listener" "web" {
load_balancer_arn = aws_lb.main.arn
port = "80"
protocol = "HTTP"
# Redirect HTTP to HTTPS
default_action {
type = "redirect"
redirect {
port = "443"
protocol = "HTTPS"
status_code = "HTTP_301"
}
}
}
resource "aws_lb_listener" "web_https" {
load_balancer_arn = aws_lb.main.arn
port = "443"
protocol = "HTTPS"
ssl_policy = "ELBSecurityPolicy-TLS-1-2-2017-01"
certificate_arn = aws_acm_certificate_validation.main.certificate_arn
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.web.arn
}
}
# Advanced routing rules
resource "aws_lb_listener_rule" "api" {
listener_arn = aws_lb_listener.web_https.arn
priority = 100
action {
type = "forward"
target_group_arn = aws_lb_target_group.api.arn
}
condition {
path_pattern {
values = ["/api/*"]
}
}
}
# ALB Security Group
resource "aws_security_group" "alb" {
name = "${var.environment}-alb-sg"
description = "Security group for Application Load Balancer"
vpc_id = aws_vpc.main.id
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "${var.environment}-alb-sg"
}
}
SSL/TLS Certificates with ACM
# Request SSL certificate
resource "aws_acm_certificate" "main" {
domain_name = var.domain_name
subject_alternative_names = ["*.${var.domain_name}"]
validation_method = "DNS"
lifecycle {
create_before_destroy = true
}
tags = {
Name = "${var.environment}-certificate"
}
}
# DNS validation (if using Route53)
data "aws_route53_zone" "main" {
name = var.domain_name
private_zone = false
}
resource "aws_route53_record" "cert_validation" {
for_each = {
for dvo in aws_acm_certificate.main.domain_validation_options : dvo.domain_name => {
name = dvo.resource_record_name
record = dvo.resource_record_value
type = dvo.resource_record_type
}
}
allow_overwrite = true
name = each.value.name
records = [each.value.record]
ttl = 60
type = each.value.type
zone_id = data.aws_route53_zone.main.zone_id
}
resource "aws_acm_certificate_validation" "main" {
certificate_arn = aws_acm_certificate.main.arn
validation_record_fqdns = [for record in aws_route53_record.cert_validation : record.fqdn]
timeouts {
create = "5m"
}
}
Auto Scaling Groups
Auto Scaling Groups automatically adjust capacity based on demand.
# Launch Template
resource "aws_launch_template" "web" {
name_prefix = "${var.environment}-web-"
image_id = data.aws_ami.amazon_linux.id
instance_type = var.web_instance_type
key_name = aws_key_pair.main.key_name
vpc_security_group_ids = [aws_security_group.web.id]
# IAM instance profile
iam_instance_profile {
name = aws_iam_instance_profile.web.name
}
# User data
user_data = base64encode(templatefile("${path.module}/scripts/web-server-setup.sh", {
environment = var.environment
}))
# EBS optimization
ebs_optimized = true
# Block device mappings
block_device_mappings {
device_name = "/dev/xvda"
ebs {
volume_size = 20
volume_type = "gp3"
delete_on_termination = true
encrypted = true
}
}
# Instance metadata service configuration
metadata_options {
http_endpoint = "enabled"
http_tokens = "required" # Require IMDSv2
}
# Monitoring
monitoring {
enabled = true
}
tag_specifications {
resource_type = "instance"
tags = {
Name = "${var.environment}-web-asg"
Environment = var.environment
Type = "web"
}
}
lifecycle {
create_before_destroy = true
}
}
# Auto Scaling Group
resource "aws_autoscaling_group" "web" {
name = "${var.environment}-web-asg"
vpc_zone_identifier = aws_subnet.private[*].id
target_group_arns = [aws_lb_target_group.web.arn]
health_check_type = "ELB"
health_check_grace_period = 300
min_size = var.web_asg_min_size
max_size = var.web_asg_max_size
desired_capacity = var.web_asg_desired_capacity
# Instance refresh for rolling updates
instance_refresh {
strategy = "Rolling"
preferences {
min_healthy_percentage = 50
}
}
launch_template {
id = aws_launch_template.web.id
version = "$Latest"
}
# Tags
tag {
key = "Name"
value = "${var.environment}-web-asg"
propagate_at_launch = true
}
tag {
key = "Environment"
value = var.environment
propagate_at_launch = true
}
lifecycle {
create_before_destroy = true
}
}
# Auto Scaling Policies
resource "aws_autoscaling_policy" "web_scale_up" {
name = "${var.environment}-web-scale-up"
scaling_adjustment = 1
adjustment_type = "ChangeInCapacity"
cooldown = 300
autoscaling_group_name = aws_autoscaling_group.web.name
}
resource "aws_autoscaling_policy" "web_scale_down" {
name = "${var.environment}-web-scale-down"
scaling_adjustment = -1
adjustment_type = "ChangeInCapacity"
cooldown = 300
autoscaling_group_name = aws_autoscaling_group.web.name
}
# CloudWatch Alarms
resource "aws_cloudwatch_metric_alarm" "web_cpu_high" {
alarm_name = "${var.environment}-web-cpu-high"
comparison_operator = "GreaterThanThreshold"
evaluation_periods = "2"
metric_name = "CPUUtilization"
namespace = "AWS/EC2"
period = "300"
statistic = "Average"
threshold = "70"
alarm_description = "This metric monitors ec2 cpu utilization"
alarm_actions = [aws_autoscaling_policy.web_scale_up.arn]
dimensions = {
AutoScalingGroupName = aws_autoscaling_group.web.name
}
}
resource "aws_cloudwatch_metric_alarm" "web_cpu_low" {
alarm_name = "${var.environment}-web-cpu-low"
comparison_operator = "LessThanThreshold"
evaluation_periods = "2"
metric_name = "CPUUtilization"
namespace = "AWS/EC2"
period = "300"
statistic = "Average"
threshold = "30"
alarm_description = "This metric monitors ec2 cpu utilization"
alarm_actions = [aws_autoscaling_policy.web_scale_down.arn]
dimensions = {
AutoScalingGroupName = aws_autoscaling_group.web.name
}
}
Container Orchestration with ECS
ECS Fargate provides serverless container hosting.
# ECS Cluster
resource "aws_ecs_cluster" "main" {
name = "${var.environment}-cluster"
# Cluster-level logging
setting {
name = "containerInsights"
value = "enabled"
}
tags = {
Name = "${var.environment}-cluster"
Environment = var.environment
}
}
# ECS Task Definition
resource "aws_ecs_task_definition" "app" {
family = "${var.environment}-app"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
cpu = 256
memory = 512
execution_role_arn = aws_iam_role.ecs_execution_role.arn
task_role_arn = aws_iam_role.ecs_task_role.arn
container_definitions = jsonencode([
{
name = "app"
image = "${aws_ecr_repository.app.repository_url}:latest"
portMappings = [
{
containerPort = 8080
hostPort = 8080
protocol = "tcp"
}
]
# Environment variables
environment = [
{
name = "NODE_ENV"
value = var.environment
},
{
name = "PORT"
value = "8080"
}
]
# Secrets from AWS Secrets Manager
secrets = [
{
name = "DB_PASSWORD"
valueFrom = aws_secretsmanager_secret.db_password.arn
}
]
# Logging
logConfiguration = {
logDriver = "awslogs"
options = {
"awslogs-group" = aws_cloudwatch_log_group.ecs_app.name
"awslogs-region" = var.aws_region
"awslogs-stream-prefix" = "ecs"
}
}
# Health check
healthCheck = {
command = ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"]
interval = 30
timeout = 5
retries = 3
startPeriod = 60
}
}
])
tags = {
Name = "${var.environment}-app-task"
Environment = var.environment
}
}
# ECS Service
resource "aws_ecs_service" "app" {
name = "${var.environment}-app-service"
cluster = aws_ecs_cluster.main.id
task_definition = aws_ecs_task_definition.app.arn
desired_count = var.app_desired_count
launch_type = "FARGATE"
# Network configuration
network_configuration {
security_groups = [aws_security_group.ecs_tasks.id]
subnets = aws_subnet.private[*].id
assign_public_ip = false
}
# Load balancer integration
load_balancer {
target_group_arn = aws_lb_target_group.api.arn
container_name = "app"
container_port = 8080
}
# Service discovery (optional)
service_registries {
registry_arn = aws_service_discovery_service.app.arn
}
# Deployment configuration
deployment_configuration {
maximum_percent = 200
minimum_healthy_percent = 100
}
# Auto-scaling integration
lifecycle {
ignore_changes = [desired_count]
}
depends_on = [
aws_lb_listener.web_https,
aws_iam_role_policy_attachment.ecs_execution_role_policy
]
tags = {
Name = "${var.environment}-app-service"
Environment = var.environment
}
}
# ECS Security Group
resource "aws_security_group" "ecs_tasks" {
name = "${var.environment}-ecs-tasks-sg"
description = "Security group for ECS tasks"
vpc_id = aws_vpc.main.id
ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
security_groups = [aws_security_group.alb.id]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "${var.environment}-ecs-tasks-sg"
}
}
# CloudWatch Log Group
resource "aws_cloudwatch_log_group" "ecs_app" {
name = "/ecs/${var.environment}-app"
retention_in_days = 30
tags = {
Name = "${var.environment}-ecs-app-logs"
Environment = var.environment
}
}
ElastiCache for Caching
# ElastiCache subnet group
resource "aws_elasticache_subnet_group" "main" {
name = "${var.environment}-cache-subnet"
subnet_ids = aws_subnet.private[*].id
tags = {
Name = "${var.environment}-cache-subnet"
}
}
# ElastiCache parameter group
resource "aws_elasticache_parameter_group" "redis" {
family = "redis7.x"
name = "${var.environment}-redis-params"
# Redis configuration parameters
parameter {
name = "maxmemory-policy"
value = "allkeys-lru"
}
}
# ElastiCache replication group
resource "aws_elasticache_replication_group" "main" {
replication_group_id = "${var.environment}-redis"
description = "Redis cluster for ${var.environment}"
# Engine configuration
engine = "redis"
engine_version = "7.0"
node_type = var.redis_node_type
port = 6379
parameter_group_name = aws_elasticache_parameter_group.redis.name
# Cluster configuration
num_cache_clusters = 2
# Network configuration
subnet_group_name = aws_elasticache_subnet_group.main.name
security_group_ids = [aws_security_group.redis.id]
# Backup configuration
snapshot_retention_limit = 5
snapshot_window = "03:00-05:00"
maintenance_window = "sun:05:00-sun:07:00"
# Security
at_rest_encryption_enabled = true
transit_encryption_enabled = true
auth_token = var.redis_auth_token
# High availability
automatic_failover_enabled = true
multi_az_enabled = true
# Notifications
notification_topic_arn = aws_sns_topic.alerts.arn
tags = {
Name = "${var.environment}-redis"
Environment = var.environment
}
}
# Redis Security Group
resource "aws_security_group" "redis" {
name = "${var.environment}-redis-sg"
description = "Security group for Redis"
vpc_id = aws_vpc.main.id
ingress {
from_port = 6379
to_port = 6379
protocol = "tcp"
security_groups = [aws_security_group.ecs_tasks.id, aws_security_group.app.id]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "${var.environment}-redis-sg"
}
}
CloudFront CDN
# S3 bucket for static assets
resource "aws_s3_bucket" "static_assets" {
bucket = "${var.environment}-${random_string.bucket_suffix.result}-static"
tags = {
Name = "${var.environment}-static-assets"
Environment = var.environment
}
}
# Bucket versioning
resource "aws_s3_bucket_versioning" "static_assets" {
bucket = aws_s3_bucket.static_assets.id
versioning_configuration {
status = "Enabled"
}
}
# Bucket public access block
resource "aws_s3_bucket_public_access_block" "static_assets" {
bucket = aws_s3_bucket.static_assets.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
# Origin Access Identity
resource "aws_cloudfront_origin_access_identity" "main" {
comment = "OAI for ${var.environment} static assets"
}
# Bucket policy for CloudFront
resource "aws_s3_bucket_policy" "static_assets" {
bucket = aws_s3_bucket.static_assets.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "AllowCloudFrontAccess"
Effect = "Allow"
Principal = {
AWS = aws_cloudfront_origin_access_identity.main.iam_arn
}
Action = "s3:GetObject"
Resource = "${aws_s3_bucket.static_assets.arn}/*"
}
]
})
}
# CloudFront Distribution
resource "aws_cloudfront_distribution" "main" {
# Origin for static assets
origin {
domain_name = aws_s3_bucket.static_assets.bucket_regional_domain_name
origin_id = "S3-${aws_s3_bucket.static_assets.id}"
s3_origin_config {
origin_access_identity = aws_cloudfront_origin_access_identity.main.cloudfront_access_identity_path
}
}
# Origin for API
origin {
domain_name = aws_lb.main.dns_name
origin_id = "ALB-${var.environment}"
custom_origin_config {
http_port = 80
https_port = 443
origin_protocol_policy = "https-only"
origin_ssl_protocols = ["TLSv1.2"]
}
}
enabled = true
is_ipv6_enabled = true
comment = "CloudFront distribution for ${var.environment}"
default_root_object = "index.html"
# Alternate domain names
aliases = [var.domain_name]
# Default cache behavior (static assets)
default_cache_behavior {
allowed_methods = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]
cached_methods = ["GET", "HEAD"]
target_origin_id = "S3-${aws_s3_bucket.static_assets.id}"
compress = true
viewer_protocol_policy = "redirect-to-https"
forwarded_values {
query_string = false
cookies {
forward = "none"
}
}
# TTL settings
min_ttl = 0
default_ttl = 3600
max_ttl = 86400
}
# API cache behavior
ordered_cache_behavior {
path_pattern = "/api/*"
allowed_methods = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]
cached_methods = ["GET", "HEAD", "OPTIONS"]
target_origin_id = "ALB-${var.environment}"
compress = true
forwarded_values {
query_string = true
headers = ["Accept", "Accept-Language", "Authorization", "CloudFront-Forwarded-Proto", "Host"]
cookies {
forward = "all"
}
}
viewer_protocol_policy = "redirect-to-https"
min_ttl = 0
default_ttl = 0
max_ttl = 0
}
# Geographic restrictions
restrictions {
geo_restriction {
restriction_type = "none"
}
}
# SSL certificate
viewer_certificate {
acm_certificate_arn = aws_acm_certificate_validation.main.certificate_arn
ssl_support_method = "sni-only"
minimum_protocol_version = "TLSv1.2_2021"
}
# Custom error pages
custom_error_response {
error_code = 404
response_code = 200
response_page_path = "/index.html"
}
# Logging
logging_config {
bucket = aws_s3_bucket.cloudfront_logs.bucket_domain_name
include_cookies = false
prefix = "cloudfront/"
}
tags = {
Name = "${var.environment}-cloudfront"
Environment = var.environment
}
}
Secrets Management
# Secrets Manager
resource "aws_secretsmanager_secret" "db_password" {
name = "${var.environment}/database/password"
description = "Database password for ${var.environment}"
recovery_window_in_days = var.environment == "prod" ? 30 : 0
tags = {
Name = "${var.environment}-db-password"
Environment = var.environment
}
}
resource "aws_secretsmanager_secret_version" "db_password" {
secret_id = aws_secretsmanager_secret.db_password.id
secret_string = var.db_password
}
# API keys secret
resource "aws_secretsmanager_secret" "api_keys" {
name = "${var.environment}/api/keys"
description = "API keys for ${var.environment}"
recovery_window_in_days = var.environment == "prod" ? 30 : 0
tags = {
Name = "${var.environment}-api-keys"
Environment = var.environment
}
}
resource "aws_secretsmanager_secret_version" "api_keys" {
secret_id = aws_secretsmanager_secret.api_keys.id
secret_string = jsonencode({
openai_api_key = var.openai_api_key
stripe_secret_key = var.stripe_secret_key
jwt_secret = var.jwt_secret
})
}
IAM Roles for ECS
# ECS Execution Role
resource "aws_iam_role" "ecs_execution_role" {
name = "${var.environment}-ecs-execution-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "ecs-tasks.amazonaws.com"
}
}
]
})
tags = {
Name = "${var.environment}-ecs-execution-role"
Environment = var.environment
}
}
# Attach AWS managed policy
resource "aws_iam_role_policy_attachment" "ecs_execution_role_policy" {
role = aws_iam_role.ecs_execution_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}
# Custom policy for secrets access
resource "aws_iam_role_policy" "ecs_secrets_policy" {
name = "${var.environment}-ecs-secrets-policy"
role = aws_iam_role.ecs_execution_role.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = [
"secretsmanager:GetSecretValue"
]
Resource = [
aws_secretsmanager_secret.db_password.arn,
aws_secretsmanager_secret.api_keys.arn
]
}
]
})
}
# ECS Task Role (for application permissions)
resource "aws_iam_role" "ecs_task_role" {
name = "${var.environment}-ecs-task-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "ecs-tasks.amazonaws.com"
}
}
]
})
tags = {
Name = "${var.environment}-ecs-task-role"
Environment = var.environment
}
}
# Task role policies
resource "aws_iam_role_policy" "ecs_task_s3_policy" {
name = "${var.environment}-ecs-task-s3-policy"
role = aws_iam_role.ecs_task_role.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
]
Resource = "${aws_s3_bucket.static_assets.arn}/*"
}
]
})
}
Monitoring and Alerting
# SNS Topic for alerts
resource "aws_sns_topic" "alerts" {
name = "${var.environment}-alerts"
tags = {
Name = "${var.environment}-alerts"
Environment = var.environment
}
}
# CloudWatch Dashboard
resource "aws_cloudwatch_dashboard" "main" {
dashboard_name = "${var.environment}-dashboard"
dashboard_body = jsonencode({
widgets = [
{
type = "metric"
x = 0
y = 0
width = 12
height = 6
properties = {
metrics = [
["AWS/ApplicationELB", "RequestCount", "LoadBalancer", aws_lb.main.arn_suffix],
[".", "TargetResponseTime", ".", "."],
[".", "HTTPCode_Target_2XX_Count", ".", "."],
[".", "HTTPCode_Target_4XX_Count", ".", "."],
[".", "HTTPCode_Target_5XX_Count", ".", "."]
]
view = "timeSeries"
stacked = false
region = var.aws_region
title = "Load Balancer Metrics"
period = 300
}
}
]
})
}
Additional Variables
Add these to your variables.tf:
variable "domain_name" {
description = "Domain name for the application"
type = string
}
variable "redis_node_type" {
description = "ElastiCache node type"
type = string
default = "cache.t3.micro"
}
variable "redis_auth_token" {
description = "Auth token for Redis"
type = string
sensitive = true
}
variable "app_desired_count" {
description = "Desired number of ECS tasks"
type = number
default = 2
}
variable "web_asg_min_size" {
description = "Minimum size of ASG"
type = number
default = 1
}
variable "web_asg_max_size" {
description = "Maximum size of ASG"
type = number
default = 5
}
variable "web_asg_desired_capacity" {
description = "Desired capacity of ASG"
type = number
default = 2
}
Deployment Strategy
- Blue-Green Deployments with ECS services
- Rolling Updates with Auto Scaling Groups
- Canary Releases using weighted routing in ALB
- Infrastructure Updates using Terraform workspaces
Security Enhancements
- WAF Integration with CloudFront and ALB
- VPC Flow Logs for network monitoring
- GuardDuty for threat detection
- Config Rules for compliance monitoring
Next Steps
You now have a production-ready architecture with:
- ✅ Load balancing and SSL termination
- ✅ Auto-scaling capabilities
- ✅ Container orchestration
- ✅ High-performance caching
- ✅ Global content delivery
- ✅ Secure secrets management
In Part 4, we'll cover:
- Terraform modules and code organization
- CI/CD integration
- Multi-environment management
- Monitoring and observability
- Cost optimization strategies
Key Takeaways:
- Use load balancers for high availability and traffic distribution
- Implement auto-scaling for cost optimization and performance
- Containerize applications for better resource utilization
- Use CDN for improved user experience globally
- Secure sensitive data with AWS Secrets Manager

