EKS Cluster: Part 15.80 – Argo Rollouts, Blue/Green

Series of blog posts show progress of updating/adding to EKS Cluster, this post covers adding Argo Rollouts.

Argo Rollouts:

Then build progressive delivery
- Install Argo Rollouts through Argo CD.
- Convert the Gateway demo Deployment into a Rollout.
- Demonstrate blue/green deployment.
- Add canary releases.
- Connect Rollouts to Gateway API.
- Add Prometheus analysis.
- Deploy an intentionally unhealthy version.
- Demonstrate automated rollback.

Phase 5: Add blue/green deployment.

Sources:
- https://argo-rollouts.readthedocs.io/en/stable/features/bluegreen
- https://argo-rollouts.readthedocs.io/en/latest
Unlike canary, users are not gradually mixed across versions. The active Service remains entirely on blue while you test green through a separate preview Service. Promotion switches the active Service selector to green.

apps/rollouts-bluegreen/
├── namespace.yaml
├── active-service.yaml
├── preview-service.yaml
└── rollout.yaml

applications/lab/workloads/
└── rollouts-bluegreen.yaml

Environment	Purpose
Active/blue	Receives normal production traffic
Preview/green	Runs the new version for testing
Promotion	Switches the active Service to green
Previous blue	Remains briefly available, then scales down

Create a new folder for 2 separate services – active & preview:

Creates the isolated namespace plus the active and preview Services. These Services will eventually point to different ReplicaSets during a blue/green update.
- rollouts-bluegreen-active
- rollouts-bluegreen-preview

aapps/rollouts-bluegreen/active-service.yaml

The active Service represents the production endpoint & Argo Rollouts modifies its selector during promotion.

apps/rollouts-bluegreen/preview-service.yaml

Sources:
- https://argo-rollouts.readthedocs.io/en/stable/features/bluegreen/
The preview Service exposes the new version before promotion. Argo Rollouts modifies its selector to target the preview ReplicaSet.
Do not add rollouts-pod-template-hash manually.
Argo Rollouts injects and updates that selector so each Service points to the correct ReplicaSet.
- The active Service remains on the stable version while the preview Service targets the new version

Create the blue/green rollout:

apps/rollouts-bluegreen/rollout.yaml

Sources:
- https://argo-rollouts.readthedocs.io/en/stable/features/bluegreen/
- https://argo-rollouts.readthedocs.io/en/stable/features/specification/
Start with blue w/One preview replica to reduce cost
Three active replicas
Manual promotion
Thirty-second old-version scale-down delay

Setting	Purpose
`activeService`	Routes normal traffic to the stable version
`previewService`	Lets you test the new version before promotion
`previewReplicaCount: 1`	Limits the preview environment’s resource usage
`autoPromotionEnabled: false`	Requires your manual approval
`scaleDownDelaySeconds: 30`	Keeps the old version briefly after switching

applications/lab/projects/lab-workloads.yaml

Minor update to add a new destination

			
======================================
spec:
  destinations:
    - namespace: rollouts-bluegreen
      server: https://kubernetes.default.svc
======================================

		

applications/lab/workloads/rollouts-bluegreen.yaml

Sources:
Ensure the annotation follows

			
======================================
metadata:
  annotations:
    # Apply this after the AppProject destination update.
    argocd.argoproj.io/sync-wave: "1"
======================================

		

Notice the path is a previous folder created just earlier that holds the new files also mentioned

			
======================================
spec:
  source:
    repoURL: https://github.com/earpjennings37/say_when_git_ops.git
    targetRevision: main
    path: apps/rollouts-bluegreen
======================================

		

Argo Rollouts dynamically adds and changes this selector. Argo CD must not remove it during self-healing.
Again see the 2 files we just created are referenced to be ignored
- apps/rollouts-bluegreen/active-service.yaml
- apps/rollouts-bluegreen/preview-service.yaml

 ======================================
 ignoreDifferences:
    - group: ""
      kind: Service
      name: rollouts-bluegreen-active
      namespace: rollouts-bluegreen
      jsonPointers:
        - /spec/selector/rollouts-pod-template-hash

    - group: ""
      kind: Service
      name: rollouts-bluegreen-preview
      namespace: rollouts-bluegreen
      jsonPointers:
        - /spec/selector/rollouts-pod-template-hash
======================================

Finally referencing the last file we created
- apps/rollouts-bluegreen/rollout.yaml

			
======================================
syncPolicy:
    managedNamespaceMetadata:
      labels:
        app.kubernetes.io/name: rollouts-bluegreen
        app.kubernetes.io/part-of: argo-rollouts-lab
        app.kubernetes.io/managed-by: argocd
======================================

		

Quick check to ensure its added:

			
======================================
kubectl get application rollouts-bluegreen   -n argocd   --watch
NAME                 SYNC STATUS   HEALTH STATUS
rollouts-bluegreen   Synced        Healthy
======================================

		

Deploy Green

Active endpoint remains blue
Preview endpoint shows green

apps/rollouts-bluegreen/rollout.yaml

Change the image preview to green
Expected state:
- Blue remains active with three pods.
- Green starts with one preview pod.
- The Rollout pauses before promotion.
- Argo CD may show Synced/Suspended, which is expected.

			
======================================
    spec:
      securityContext:
        runAsNonRoot: true
      containers:
        - name: rollouts-demo
          image: argoproj/rollouts-demo:blue
          imagePullPolicy: IfNotPresent
======================================

		

Compare blue & green:

Open:
- Active production: http://localhost:8082
- Green preview: http://localhost:8083
Expected:
- Port 8082 displays blue.
- Port 8083 displays green.

			
======================================
kubectl argo rollouts get rollout rollouts-bluegreen \
  -n rollouts-bluegreen \
  --watch
======================================

		

Notice how blue is stable & active while green is isoalated is preview & rollout is paused for manual approval… do NOT promote just yet. Hold it… wait….. HOLD THE LINE!!

			
======================================
kubectl port-forward \
  -n rollouts-bluegreen \
  service/rollouts-bluegreen-active \
  8082:80
======================================
kubectl port-forward \
  -n rollouts-bluegreen \
  service/rollouts-bluegreen-preview \
  8083:80
======================================

		

Annotate Blue ReplicaSet:

			
======================================
kubectl annotate replicaset \
  rollouts-bluegreen-7d954647f4 \
  -n rollouts-bluegreen \
  troubleshooting.reconcile-at="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --overwrite
======================================
kubectl get replicasets,pods \
  -n rollouts-bluegreen \
  --watch
======================================
kubectl argo rollouts get rollout rollouts-bluegreen \
  -n rollouts-bluegreen
======================================

		

Promote Green

Active Service switches to green
Old blue ReplicaSet scales down after the safety delay

Promote & transition

Expected sequence:
- Green scales from one to three pods.
- Active Service switches from the blue hash to the green hash.
- Green becomes stable, active.
- Blue remains running for the 30-second safety delay.
- Blue scales to zero.

			
======================================
kubectl argo rollouts promote rollouts-bluegreen \
  -n rollouts-bluegreen
======================================
kubectl argo rollouts get rollout rollouts-bluegreen \
  -n rollouts-bluegreen \
  --watch
======================================

		

Final Verification

Turn up, we have
- Green: 3/3, stable and active
- Blue: 0/0, scaled down
- ArgoCD: Synced & Healthy
To summarize:
- Blue remained active while green was tested through the preview Service.
- Green was manually promoted.
- The active Service switched to green.
- Green scaled from one preview pod to three active pods.
- Blue remained briefly during the safety delay.
- Blue then scaled to zero.
- Argo CD returned to Synced/Healthy.

			
======================================
kubectl get replicasets \
  -n rollouts-bluegreen
NAME                            DESIRED   CURRENT   READY   AGE
rollouts-bluegreen-576485f845   3         3         3       40m
rollouts-bluegreen-7d954647f4   0         0         0       55m
======================================
kubectl get application rollouts-bluegreen \
  -n argocd \
  -o custom-columns='NAME:.metadata.name,SYNC:.status.sync.status,HEALTH:.status.health.status'
  NAME                 SYNC     HEALTH
rollouts-bluegreen   Synced   Healthy
======================================

		

			
======================================
kubectl argo rollouts get rollout rollouts-bluegreen \
  -n rollouts-bluegreen
  Name:            rollouts-bluegreen
Namespace:       rollouts-bluegreen
Status:          ✔ Healthy
Strategy:        BlueGreen
Images:          argoproj/rollouts-demo:green (stable, active)
Replicas:
  Desired:       3
  Current:       3
  Updated:       3
  Ready:         3
  Available:     3
NAME                                            KIND        STATUS        AGE  INFO
⟳ rollouts-bluegreen                            Rollout     ✔ Healthy     59m  
├──# revision:2                                                                
│  └──⧉ rollouts-bluegreen-576485f845           ReplicaSet  ✔ Healthy     44m  stable,active
│     ├──□ rollouts-bluegreen-576485f845-8skqn  Pod         ✔ Running     10m  ready:1/1
│     ├──□ rollouts-bluegreen-576485f845-kqjkf  Pod         ✔ Running     10m  ready:1/1
│     └──□ rollouts-bluegreen-576485f845-s6sbj  Pod         ✔ Running     10m  ready:1/1
└──# revision:1                                                                
   └──⧉ rollouts-bluegreen-7d954647f4           ReplicaSet  • ScaledDown  59m  
======================================

		

EKS Cluster: Part 15.80 – Argo Rollouts, Blue/Green

Published by EarpMoonWater

Leave a Reply Cancel reply

Share this:

Published by EarpMoonWater

Leave a Reply Cancel reply