Advanced Track Do this after finishing Chapters 01-14.

Estimated Time

  • Reading: 30-40 min
  • Lab: 60-90 min
  • Quiz: 15-20 min

Prerequisites

  • Core track (Chapters 01-14) completed.
  • GitOps promotion and observability workflows available.

Source Code References

  • canary.example.yaml Members

Sign in to view source code.

What You Will Produce

A go/no-go evidence package: rollout results, remediation notes, and explicit rollback conditions.

Investigation

Treat canary metrics as the definitive proof of release safety.

Safe investigation sequence:

  1. Verify Traffic Split: Check the current traffic distribution between the stable and canary versions.
  2. Monitor Canary Metrics: Analyze the latency and error rates specifically for the canary pods.
  3. Compare with Stable: Compare canary performance against the baseline of the stable version.
  4. Inspect Flagger Events: Review the Flagger logs or Kubernetes events to see why an analysis is progressing or stalled.

Containment

Containment is built directly into the progressive delivery engine.

Containment steps:

  1. Automatic Halt: Flagger automatically stops the traffic shift if analysis thresholds are violated.
  2. Immediate Revert: Traffic is shifted back to 100% stable version instantly upon failure detection.
  3. Isolate Canary Pods: Keep the failed canary pods running (but without traffic) for debugging.
  4. Fix Upstream: Resolve the bug in Git and push a new version to restart the canary cycle.

The goal is “automated safety,” where the system protects the user from bad code.

Pause and Predict: What automated guardrail would have prevented this incident entirely?