Investigation & Containment

Investigation

Start by proving identity, not by rebuilding again.

Safe investigation sequence:

  1. Compare digests: Compare staging and production image digests directly.
  2. Inspect Git evidence: Review the promotion commit and metadata in the repository.
  3. Confirm Flux actions: Confirm what ImagePolicy selected and what ImageUpdateAutomation wrote back to Git.
  4. Determine lineage: Decide whether the deployment was a real promotion or a new build wearing a familiar name.

Containment

Containment restores one trustworthy artifact path.

Containment steps:

  1. Revert to known-good: Revert to the last known-good production promotion commit in Git.
  2. Flux Reconcile: Let Flux reconcile the previous, stable digest.
  3. Verify identity: Verify that the deployed workload matches the intended immutable artifact.
  4. Re-run correctly: Re-run the promotion only after the artifact lineage is clear again.

Pause and Predict: What automated guardrail would have prevented this incident entirely?