terraform – Page 6

Karpenter: Dynamically launches compute based on pending workload needs rather than relying only on fixed managed node groups.

Series of 3 blog posts show progress of updating/adding to EKS Cluster, this post covers adding Karpenter. This in particular covers the infrastructure AWS & Terraform portion of Karpenter.

infra/Terraform repository

karpenter.tf
├── controller IAM role
├── controller IAM policy
├── Pod Identity association
├── Karpenter EC2 node role
├── EKS node access entry
├── SQS interruption queue
├── EventBridge rules
├── queue policy
└── subnet/security-group discovery tags

vpc.tf

Purpose:
- Adds karpenter.sh/discovery: say-when-east to the east public subnets.
- Allows Karpenter to discover eligible subnets dynamically rather than hardcoding subnet IDs through this tag and selects the subnet with the most available IP addresses in the required Availability Zone
Sources

			
==================
   # Allows the Karpenter EC2NodeClass to discover both east subnets.
  public_subnet_tags = {
    "karpenter.sh/discovery" = local.eks_east_name
  }
==================

		

eks.tf

Purpose
- Adds the Karpenter discovery tag to the EKS node security group.
  - Only the node security group should receive the discovery tag
- Karpenter attaches every security group matching its selector, so tagging multiple EKS security groups could attach more groups than intended
- Allows the EC2NodeClass to discover the correct security group.
Sources:
- Karpenter NodeClasses: security-group selection
- Terraform AWS EKS module

			
==================
    node_security_group_tags = {
    "karpenter.sh/discovery" = local.eks_east_name
  }
==================

		

karpenter.tf

Purpose:
- Creates the Karpenter controller IAM role.
- Creates its EKS Pod Identity association.
- Creates the SQS interruption queue and EventBridge rules.
- Supplies node-provisioning permissions.
- Creates the stable EC2 node instance profile.
- Adds the missing iam:ListInstanceProfiles permission.

Sources
- Terraform EKS Karpenter submodule v20.37.2
- AWS EKS Pod Identity
- Assign an IAM role through EKS Pod Identity
  - IAM role contains permissions, but ec2 cant use role directly, so ec2 receives the role from the iam instance profile.
- AWS EKS Pod Identity trust policy
- AWS IAM instance profiles
  - Instance profile is the bridge between karpenter ec2 & my existing eks worker-node iam role.
- Terraform Module: Karpenter – Mintify
  - Expresses the intent w/one flag, while the module contains AWS resources

The module internally creates interruption handling & sub-module contains resources similar to below. It creates the SQS queue, queue policy, EventBridge rules, and EventBridge targets when native interruption handling is enabled.

			
==================
resource "aws_sqs_queue" "this" {
  # Interruption queue
}
resource "aws_sqs_queue_policy" "this" {
  # Permits EventBridge to send messages
}
resource "aws_cloudwatch_event_rule" "this" {
  # Spot interruption, rebalance and EC2 lifecycle rules
}
==================

		

			
==================
enable_spot_termination = true
==================

Karpenter introduced this permission for its instance-profile garbage-collection controller. Resource = "*" is expected for this IAM list operation

			
======================
resource "aws_iam_role_policy" "karpenter_instance_profile_gc" {
  provider = aws.east
  name = "KarpenterInstanceProfileGarbageCollection"
  role = module.karpenter_east.iam_role_name
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid      = "AllowListingInstanceProfiles"
        Effect   = "Allow"
        Action   = "iam:ListInstanceProfiles"
        Resource = "*"
      }
    ]
  })
}
======================

		