The data module

What we’re building

ShopMicro needs a PostgreSQL database. On a laptop that’s a container; in production it’s a managed database — backups, patching, and failover handled by the cloud — living privately inside the same network as the cluster, reachable only from inside.

This lesson defines the data module’s interface and builds the AWS RDS implementation. Like network, cluster, and iam, it has one interface on every cloud so the live/ units wire it identically:

• in: name, network_id, subnet_ids, db_name (default shopmicro)
• out: db_host, db_port, db_name, db_user, db_password (sensitive)

The next lesson, managed Postgres, implements the same interface on GCP Cloud SQL and Azure Database, and turns those five outputs into the connection secret ShopMicro consumes. Here we establish the shape and the AWS version, taking a dependency "network" so the database lands in the private subnets.

Why

Managed Postgres over self-hosted is the easy call for a capstone: nobody learns anything running their own postgres StatefulSet with hand-rolled backups. The interesting decisions are where it lives and how the password travels.

It lives in the private subnets with no public address, so the only path to it is from inside the network — the cluster’s pods. That’s a security boundary you get from the network module for free, and it’s why the module takes network_id and subnet_ids rather than provisioning its own.

The password is generated, never written by a human, and surfaced as a sensitive output. Terraform marks it so it’s redacted from plan/apply logs and the console; the only consumer is the ShopMicro unit, which reads it straight from state to build a Kubernetes Secret. No password ever appears in a variables file or a commit.

Pros & cons

Managed Postgres vs a self-hosted StatefulSet

• Pros: the cloud handles backups, minor-version patching, storage growth, and failover; one resource block instead of an operator, volumes, and a backup CronJob; it’s what you’d actually run in production.
• Cons: it costs more than a pod, you’re on the cloud’s version cadence, and each cloud’s managed offering has its own quirks (parameter groups, flags, SKUs) — which is exactly the per-cloud difference the next lesson has to absorb.

Generated password as sensitive output vs an externally-managed secret

• Pros: random_password + a sensitive output is self-contained — no human ever sees it, and the ShopMicro unit reads it directly from the dependency; zero manual steps.
• Cons: the plaintext lives in Terraform state, so the state backend must be encrypted and locked (which ours is — S3 + encryption in the remote state setup). A dedicated secrets manager is the next step up, noted at the end of the course.

Set it up

The AWS implementation is a subnet group over the private subnets, a security group that only admits the VPC, a generated password, and the RDS instance itself.

1. modules/aws/data/variables.tf

variable "name"       { type = string }
variable "network_id" { type = string } # the VPC id
variable "subnet_ids" { type = list(string) }

variable "db_name" {
  type    = string
  default = "shopmicro"
}

2. modules/aws/data/main.tf

data "aws_vpc" "this" {
  id = var.network_id
}

resource "aws_db_subnet_group" "this" {
  name       = "${var.name}-db"
  subnet_ids = var.subnet_ids
}

resource "aws_security_group" "db" {
  name   = "${var.name}-db"
  vpc_id = var.network_id

  ingress {
    description = "PostgreSQL from within the VPC"
    from_port   = 5432
    to_port     = 5432
    protocol    = "tcp"
    cidr_blocks = [data.aws_vpc.this.cidr_block]
  }
}

resource "random_password" "db" {
  length  = 24
  special = false # keep it URL-safe for the connection string
}

resource "aws_db_instance" "this" {
  identifier     = var.name
  engine         = "postgres"
  engine_version = "16"          # prefix; RDS resolves the latest 16.x
  instance_class = "db.t4g.micro"

  allocated_storage     = 20
  max_allocated_storage = 100    # storage autoscaling ceiling

  db_name  = var.db_name
  username = "shopmicro"
  password = random_password.db.result

  db_subnet_group_name   = aws_db_subnet_group.this.name
  vpc_security_group_ids = [aws_security_group.db.id]
  publicly_accessible    = false

  storage_encrypted   = true
  skip_final_snapshot = true     # a teaching platform, not prod data
}

publicly_accessible = false plus a security group scoped to the VPC CIDR is the private-only boundary. engine_version = "16" is a prefix — RDS picks the latest available 16.x, which is fine when auto_minor_version_upgrade stays on.

3. modules/aws/data/outputs.tf

The five interface outputs, with the password marked sensitive.

output "db_host" { value = aws_db_instance.this.address }
output "db_port" { value = aws_db_instance.this.port }
output "db_name" { value = aws_db_instance.this.db_name }
output "db_user" { value = aws_db_instance.this.username }

output "db_password" {
  value     = random_password.db.result
  sensitive = true
}

4. live/aws/data/terragrunt.hcl

The unit depends on network and passes the private subnets — identical in shape to what the gcp/azure units will do next lesson.

include "root" { path = find_in_parent_folders("root.hcl") }
terraform { source = "../../../modules/aws/data" }

dependency "network" { config_path = "../network" }

inputs = {
  name       = "clouddeploy"
  network_id = dependency.network.outputs.network_id
  subnet_ids = dependency.network.outputs.private_subnet_ids
  db_name    = "shopmicro"
}

Verify

Plan first — the sensitive password should already be redacted — then apply and read the outputs.

cd live/aws/data
terragrunt plan

Plan: 4 to add, 0 to change, 0 to destroy.
  # aws_db_instance.this will be created
  + password = (sensitive value)
  + address  = (known after apply)
  ...

terragrunt apply

# non-sensitive outputs print normally
terragrunt output db_host
# => "clouddeploy.abc123xyz.us-east-1.rds.amazonaws.com"
terragrunt output db_port
# => 5432

# the password is redacted unless you explicitly ask for the raw value
terragrunt output db_password
# => <sensitive>

Confirm it’s genuinely private — a connection from outside the VPC must fail, and one from a pod inside must succeed:

# from your laptop (outside the VPC): should hang / refuse
psql "postgres://shopmicro@$(terragrunt output -raw db_host):5432/shopmicro" -c '\l'
# => timeout — there is no public route, by design

# from a pod inside the cluster: reachable
kubectl run pg --rm -it --restart=Never --image=postgres:16 -- \
  psql "postgres://shopmicro:$(terragrunt output -raw db_password)@$(terragrunt output -raw db_host):5432/shopmicro" -c '\conninfo'
# => You are connected to database "shopmicro" ...

That inside-succeeds / outside-fails pair is the whole security story of this module. Then run terragrunt run --all plan from live/aws/ to confirm data slots in after network with no drift.

Check your understanding:

1. Why does the module take network_id and subnet_ids as inputs instead of creating its own subnets?
2. What does marking db_password as sensitive actually change — and what does it not protect (where does the plaintext still live)?
3. What two settings together make the database reachable only from inside the network?
4. skip_final_snapshot = true is fine here but dangerous in production. Why, and what would you change for real data?

Recap

We defined the data interface — name, network_id, subnet_ids, db_name in; db_host, db_port, db_name, db_user, db_password (sensitive) out — and built the AWS RDS implementation: a private, encrypted PostgreSQL instance in the network’s private subnets, with a generated password surfaced only as a redacted output and a dependency "network" placing it correctly.

One cloud down, the interface proven. Next, Managed Postgres → implements the same five inputs and outputs on GCP Cloud SQL and Azure Database for PostgreSQL, then assembles those outputs into the connection secret ShopMicro will consume.