Checkpoint B: Your Runtime Safety Net

You have completed Chapters 01-09. Before moving into promotion and operations, pause and look at the runtime constraints you have built.

This page is a consolidation — not new material. Its purpose is to show how the four hardening chapters (06-09) work together to contain failure.


What You Have Built

Your workloads now run inside a layered safety net. Each layer limits blast radius in a different direction:

                ┌─────────────────────────────────────┐
                │   Chapter 06: Network Policies      │
                │   Who can talk to whom              │
                ├─────────────────────────────────────┤
                │   Chapter 07: Security Context      │
                │   How pods run (non-root, read-only)│
                ├─────────────────────────────────────┤
                │   Chapter 08: Resource Management   │
                │   How much CPU and memory           │
                ├─────────────────────────────────────┤
                │   Chapter 09: Availability          │
                │   How many replicas, when           │
                └─────────────────────────────────────┘
                  ▲
                  │ Workloads deployed via Chapters 01-05

How The Layers Interact

The layers are not independent — they compose:

  • A pod hardened by security context (Ch 07) runs within the network boundaries set by network policies (Ch 06)
  • Resource requests (Ch 08) are what the HPA (Ch 09) measures CPU utilisation against, and what the kubelet compares usage with when a node runs out of memory: pods using more than they requested are evicted first, then by priority and by how far above their requests they are (the production backend requests 128Mi of memory: kubectl --context kind-sre-control-plane -n production get deploy backend -o jsonpath='{.spec.template.spec.containers[0].resources.requests.memory}')
  • Network policies (Ch 06) decide whether Prometheus may reach the metrics port: in develop, the ingress policy allow-backend-metrics-from-observability admits the observability namespace on port 8080 (kubectl --context kind-sre-control-plane -n develop get networkpolicy allow-backend-metrics-from-observability). Without it, scraping the backend stops - which is why Chapter 11 builds on these boundaries

The Guardrails You Have in Place

GuardrailSourceWhat it limitsWhat it does not
Default-deny NetworkPolicyChapter 06connections nobody wrote down - a taken-over pod reaches only what its policies allowtraffic the policies do allow; on kind, DNS to CoreDNS on the other node
runAsNonRoot: trueChapter 07root inside the container - changing root-owned files, the privileges many exploits assumewhat the app’s own user may do
readOnlyRootFilesystem: trueChapter 07changing the image’s fileswriting to the declared, size-limited volumes
Resource requests and limitsChapter 08one container taking the node’s memory; memory over the limit is OOMKilled, CPU over it throttleda leak - it only reaches the limit later with a bigger one
Two replicas (minReplicas: 2 in staging and production)Chapter 09a crashed pod - the other one keeps servinglosing the node - both replicas run on the platform’s single worker
Rolling update strategy (maxSurge: 1, maxUnavailable: 0)Chapter 09a rollout stopping the last serving pod - a new pod is Ready before an old one stops, even with one replicaa new version that is Ready but broken
PodDisruptionBudgetChapter 09how many pods a drain may evict at once: with two replicas and minAvailable: 1, the second eviction waits while the first replacement is Pendingroom for the replacements; a single replica with maxUnavailable: 1, or a pod with no budget, is evicted completely; a failed worker asks no budget at all

Self-Check

You are ready for Chapter 10 when you can answer without looking:

  • Why is default-deny safer than allow-all, even though it creates more up-front work?
  • What is the standard pattern for writable /tmp when readOnlyRootFilesystem: true is enforced?
  • What evidence do you need before raising a pod’s memory limit?
  • Why do staging and production run two replicas while develop runs one - and what do two replicas on one node not protect against?
  • A PDB blocks a node drain with ALLOWED DISRUPTIONS 0. Which three causes do you check before you change anything - too few healthy pods, a budget as strict as the replica count, or replacements with nowhere to run?

If any of these are unclear, revisit the relevant chapter before moving forward.


What Comes Next

You can now deliver code safely (Ch 01-05) and your workloads have runtime constraints (Ch 06-09). The next block (Chapters 10-15) first moves a release safely from develop to production, then shifts to what happens when something goes wrong:

  • Chapter 10 — Version Promotion: how the image tested in staging reaches production without being rebuilt
  • Chapter 11 — Observability: how you see what is happening
  • Chapter 12 — Backup and Restore: how you recover from data loss
  • Chapter 13 — Controlled Chaos: how you rehearse failure
  • Chapter 14 — AI-Assisted SRE Guardian: how you route incidents safely
  • Chapter 15 — 24/7 Production SRE: how humans coordinate response