Investigation

Start by treating state and plan history as evidence, not memory.

Safe investigation sequence:

1. Identify every job: List every plan and apply job that touched the same environment.
2. Compare artifacts: Compare the reviewed plan artifact with a fresh plan against current state.
3. Confirm stale assumptions: Confirm whether the later apply ran from assumptions older than the current state.
4. Trace the gap: Identify the workflow gap that allowed stale intent to remain executable.

The root cause here is usually workflow design, not Terraform syntax.

Containment

Containment starts by stopping the overlap and restoring state trust.

Containment steps:

1. Pause concurrent applies: Stop all automated apply pipelines for that environment.
2. Generate a fresh plan: Create a new plan from the current state.
3. Review the corrective diff: Review only the changes required to fix the drift.
4. Apply once: Apply only once from the fresh, reviewed plan.

Only after state is trustworthy again should the team tune concurrency, approvals, or destroy policy.

Pause and Predict: What automated guardrail would have prevented this incident entirely?