EKS, GKE, AKS

What we’re building

The EKS implementation proved the cluster interface works on one cloud. Now we implement it twice more — GKE (modules/gcp/cluster) and AKS (modules/azure/cluster) — behind the same cluster_name, cluster_endpoint, cluster_ca outputs, then do the thing those three outputs exist for: build a kubeconfig and configure the kubernetes and helm providers so the platform layer (Datadog, Keycloak, GrowthBook) can deploy onto any of the three clusters with identical code.

When we’re done, live/gcp/cluster and live/azure/cluster will carry the same inputs block as live/aws/cluster — abstract node_size, network ids from a dependency — and the provider configuration that turns cluster outputs into a working helm_release will be the same shape on every cloud.

Why

The three managed-Kubernetes services diverge more than the networks did — in node pools, in identity, and especially in how you authenticate to the API server:

• EKS — auth via an exec plugin: aws eks get-token.
• GKE — auth via a short-lived OAuth token from google_client_config.
• AKS — auth via the cluster’s kube_config (certificates) or an exec plugin (kubelogin) for AAD clusters.

But the inputs to every one of those auth methods are the same three values: the cluster’s name, its endpoint, and its CA. That’s why the interface stops at those three outputs — they’re the common denominator of “how do I talk to this cluster,” and each cloud’s provider block assembles them differently. Keep the interface at three outputs and the platform layer above never learns which cloud it’s on.

Pros & cons

Provider auth from cluster_endpoint + cluster_ca + an exec/token plugin vs. writing a kubeconfig file to disk

• Pros: no secret kubeconfig file lands on disk or in state; the provider fetches a fresh token each run; the same three outputs configure the provider on all three clouds.
• Cons: the machine running Terragrunt needs the cloud CLI (or kubelogin) installed for the exec plugin to work. In CI that’s an explicit setup step — which we make visible rather than assume.

Managing node pools as a separate resource (GKE) vs. inline in the cluster (AKS)

• Pros: GKE’s separate google_container_node_pool lets you change nodes without recreating the control plane — the recommended pattern. AKS’s required inline default_node_pool keeps a minimal cluster in one resource.
• Cons: the two clouds simply don’t agree on the shape, so the module code differs. The interface hides that: both still take node_count/node_size and emit the same three outputs.

Set it up

1. modules/gcp/cluster/ — GKE

The same variables.tf and node_size map (mapped to GCP machine types), a VPC-native cluster referencing the network’s secondary ranges by name (pods, services), and a separately managed node pool.

locals {
  machine_type = {
    small  = "e2-standard-2"
    medium = "e2-standard-4"
    large  = "e2-standard-8"
  }[var.node_size]
}

resource "google_container_cluster" "this" {
  name       = var.name
  location   = "us-central1"
  network    = var.network_id
  subnetwork = var.subnet_ids[0]

  # Manage nodes via a separate pool, not the default one.
  remove_default_node_pool = true
  initial_node_count       = 1

  ip_allocation_policy {
    cluster_secondary_range_name  = "pods"
    services_secondary_range_name = "services"
  }
}

resource "google_container_node_pool" "this" {
  name       = "${var.name}-default"
  cluster    = google_container_cluster.this.id
  node_count = var.node_count

  node_config {
    machine_type = local.machine_type
    oauth_scopes = ["https://www.googleapis.com/auth/cloud-platform"]
  }
}

GKE’s endpoint comes back without a scheme and its CA is under master_auth. outputs.tf normalizes to the interface:

output "cluster_name" {
  value = google_container_cluster.this.name
}

output "cluster_endpoint" {
  value = "https://${google_container_cluster.this.endpoint}"
}

output "cluster_ca" {
  value = google_container_cluster.this.master_auth[0].cluster_ca_certificate
}

2. modules/azure/cluster/ — AKS

AKS requires an inline default_node_pool and a managed identity. The cluster lands in the resource group derived from the network. node_size maps to Azure VM sizes.

locals {
  vm_size = {
    small  = "Standard_D2s_v3"
    medium = "Standard_D4s_v3"
    large  = "Standard_D8s_v3"
  }[var.node_size]
}

resource "azurerm_kubernetes_cluster" "this" {
  name                = var.name
  location            = "eastus"
  resource_group_name = "${var.name}-rg" # created by the network module
  dns_prefix          = var.name

  default_node_pool {
    name           = "default"
    node_count     = var.node_count
    vm_size        = local.vm_size
    vnet_subnet_id = var.subnet_ids[0]
  }

  identity {
    type = "SystemAssigned"
  }
}

AKS exposes everything through kube_config. outputs.tf:

output "cluster_name" {
  value = azurerm_kubernetes_cluster.this.name
}

output "cluster_endpoint" {
  value = azurerm_kubernetes_cluster.this.kube_config[0].host
}

output "cluster_ca" {
  value = azurerm_kubernetes_cluster.this.kube_config[0].cluster_ca_certificate
}

Three clouds, three very different resources — one identical set of output names.

3. The live/ units — same inputs, different source

live/gcp/cluster/terragrunt.hcl (Azure identical but for source):

include "root" {
  path = find_in_parent_folders("root.hcl")
}

terraform {
  source = "../../../modules/gcp/cluster"
}

dependency "network" {
  config_path = "../network"

  mock_outputs = {
    network_id         = "projects/p/global/networks/mock"
    private_subnet_ids = ["projects/p/regions/us-central1/subnetworks/mock"]
  }
  mock_outputs_allowed_terraform_commands = ["plan", "validate"]
}

inputs = {
  name       = "clouddeploy"
  network_id = dependency.network.outputs.network_id
  subnet_ids = dependency.network.outputs.private_subnet_ids
  node_count = 2
  node_size  = "small"
}

That inputs block is character-for-character the AWS one from last lesson. The interface earned that.

4. Building a kubeconfig / the kubernetes + helm providers

This is what the three outputs are for. The platform layer (built in later modules) runs Helm through the Terraform helm provider, which needs the kubernetes provider configured from the cluster. That configuration differs only in the auth plugin per cloud — endpoint and CA are always cluster_endpoint and base64decode(cluster_ca).

AWS — exec aws eks get-token:

provider "kubernetes" {
  host                   = var.cluster_endpoint
  cluster_ca_certificate = base64decode(var.cluster_ca)
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "aws"
    args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
  }
}

provider "helm" {
  kubernetes {
    host                   = var.cluster_endpoint
    cluster_ca_certificate = base64decode(var.cluster_ca)
    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      command     = "aws"
      args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
    }
  }
}

GCP — a short-lived token from google_client_config:

data "google_client_config" "default" {}

provider "kubernetes" {
  host                   = var.cluster_endpoint
  cluster_ca_certificate = base64decode(var.cluster_ca)
  token                  = data.google_client_config.default.access_token
}

Azure — exec kubelogin (or the cluster’s cert-based kube_config):

provider "kubernetes" {
  host                   = var.cluster_endpoint
  cluster_ca_certificate = base64decode(var.cluster_ca)
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "kubelogin"
    args        = ["get-token", "--server-id", var.cluster_name]
  }
}

Look at what’s constant across all three: host = var.cluster_endpoint, cluster_ca_certificate = base64decode(var.cluster_ca). Only the auth stanza changes — and it changes because the clouds authenticate differently, which is a genuine difference the interface can’t paper over and shouldn’t try to. Everything the helm_release does above this line is identical.

Verify

Plan each cluster from its live/ tree — dependencies resolve against mock network outputs on a clean plan:

terragrunt run --all plan --working-dir live/gcp
terragrunt run --all plan --working-dir live/azure

Confirm all three clouds emit the same three output names — the interface holding across EKS, GKE, and AKS:

for c in aws gcp azure; do
  echo "== $c =="
  terragrunt output -json --working-dir live/$c/cluster | jq 'keys'
done

== aws ==
[ "cluster_ca", "cluster_endpoint", "cluster_name" ]
== gcp ==
[ "cluster_ca", "cluster_endpoint", "cluster_name" ]
== azure ==
[ "cluster_ca", "cluster_endpoint", "cluster_name" ]

Now the real proof — apply a cluster, generate its kubeconfig, and reach it with kubectl. On GCP:

cd live/gcp/cluster && terragrunt apply
gcloud container clusters get-credentials clouddeploy --region us-central1
kubectl get nodes

NAME                                       STATUS   ROLES    AGE   VERSION
gke-clouddeploy-default-abc1-...           Ready    <none>   2m    v1.33.x
gke-clouddeploy-default-abc1-...           Ready    <none>   2m    v1.33.x

Ready nodes reached through a kubeconfig built from cluster_endpoint and cluster_ca — the same two outputs, on a different cloud. The platform layer can now Helm-deploy onto any of the three identically.

Check your understanding:

1. All three provider blocks share host and cluster_ca_certificate but differ in one stanza. Which stanza, and why is that difference the one thing the interface deliberately refuses to hide?
2. GKE’s raw endpoint output has no scheme and AKS exposes its host under kube_config. Where does that normalization happen so consumers see a uniform cluster_endpoint?
3. GKE manages its node pool as a separate resource while AKS requires it inline. How does the cluster interface keep that structural difference from reaching the live/ units?
4. A CI runner fails configuring the helm provider for EKS with “aws: command not found.” From the exec block, explain the root cause and the fix.

Recap

We implemented the cluster interface a second and third time — a VPC-native GKE cluster with a separately managed node pool, and an AKS cluster with an inline default pool and managed identity — behind the same cluster_name, cluster_endpoint, cluster_ca outputs. Then we turned those three outputs into a kubeconfig and into kubernetes/helm provider configuration whose only per-cloud variation is the auth plugin. The live/gcp/cluster and live/azure/cluster units share the AWS unit’s exact inputs, and Terragrunt orders network-then-cluster on all three.

Networks and clusters are done on every cloud, and Helm can now reach each one. But a cluster that can run workloads still needs to grant those workloads cloud access — without static keys. That’s the next layer, where the three clouds diverge hardest: Identity & IAM →.