Guardrails-First Course Materials

This course teaches production-grade Kubernetes and SRE practice through incidents, guardrails, and repeatable workflows.

The goal is not to memorize tools. The goal is to learn how to keep systems safe when pressure, ambiguity, and AI-assisted speed all show up at the same time.

Who This Is For

  • platform engineers moving from “it works” to “it survives mistakes”
  • DevOps engineers who want stronger operating discipline, not more tooling hype
  • SREs who want concrete labs, guardrails, and incident-shaped lessons

How the Course Works

Each chapter is built around one production failure pattern:

  • what broke
  • why the shortcut looked reasonable
  • how the investigation becomes confusing
  • which guardrail restores a safe operating path

From Chapter 02 on, every core chapter includes:

  • a written incident walkthrough
  • a hands-on lab
  • a quiz to confirm the operating rule
  • runbooks or scorecards where the topic needs them

Chapter 00 is the lab setup itself, and Chapter 01 sets the rules the rest of the course applies; both end with a quiz.

This course does not only teach how to operate Kubernetes around applications. It also shows what a production-ready Kubernetes application should look like so rollout safety, observability, GitOps reconciliation, and incident response work correctly in the first place.

The course uses the SafeOps reference applications as concrete examples:

  • safeops-course/backend, a small production-shaped Go API with health probes, metrics, tracing hooks, chaos endpoints (opt-in, off in production), and OpenAPI/Swagger support
  • safeops-course/frontend, a Vue-based frontend with container hardening and runtime config injection

Their Kubernetes manifests live in the platform repository as Kustomize overlays per environment.

Many of the application patterns in those reference apps are inspired by Podinfo by Stefan Prodan. What the SafeOps apps implement:

  • readiness and liveness probes
  • graceful shutdown on interrupt signals
  • config reload from a mounted directory (backend CONFIG_PATH, shown at /configs); the platform itself passes feature flags from a ConfigMap as environment variables
  • Prometheus and OpenTelemetry instrumentation
  • structured logging
  • 12-factor configuration
  • fault injection for safe drills
  • multi-arch images scanned for CVEs before they are published, then signed, with an SBOM and build provenance attached

Video assets are optional. The written lesson remains the primary source of truth, and the video should make the same lesson easier to absorb, not replace the material.

  1. Start with Intro: AI as a Very Well-Read Junior Engineer.
  2. Set up your lab in Chapter 00 - every later lab runs on it.
  3. Go through Chapters 01-15 in order.
  4. Run the lab before moving to the next chapter.
  5. Use the quiz to confirm the main guardrail rule before continuing.
  6. Move to the advanced chapters only after the core path feels operationally natural.

Tracks

Core track:

  • Chapter 00: your lab environment
  • Chapters 01-15 covering platform foundations, GitOps, CI/CD, security, observability, reliability, and on-call discipline

Advanced track:

  • Chapter 16: Admission Policy Guardrails
  • Chapter 17: Supply Chain Security
  • Chapter 18: Rollback and Data Migrations
  • Chapter 19: Progressive Delivery (Canary with Traefik + Flagger)
  • Chapter 20: Platform Lifecycle
  • Chapter 21: Credential & Access Lifecycle
  • Chapter 22: Disaster Recovery
  • Chapter 23: Capstone - Prove and Hand Over Your Platform

Reference appendices:

  • Appendix: Local Development Environment
  • Appendix: DNS and TLS Automation

References

Chapter 00: Your Lab Environment

Do this chapter first; Chapter 01, which has almost no commands, comes once your cluster is green. Is This Course for You? This course teaches production guardrails, not the basics underneath them. You should already be …

Chapter 01: Blast Radius & the Shape of Safety

This chapter has almost no commands: the story shows a few, and the only thing you set up is a hook for your AI agent (section 5). The hands-on work starts in Chapter 02. Learning Objectives By the end of this chapter, …

Chapter 02: Infrastructure as Code with Terraform

Explain what Terraform’s state is and how a plan is made from the code, the state and the real objects Show why two independent states for one infrastructure break it - two people or jobs, each with their own local …

Chapter 03: GitOps with Flux

Explain what you gain when a controller keeps the cluster equal to Git - and what that means for a change made by hand Trace any Flux-managed object to the Flux Kustomization and the Git file it comes from Recognise a …

Chapter 04: Secrets Management (SOPS + Age)

Encrypt a Secret with SOPS and age before it is committed, and trace it to the Secret Flux applies in the cluster Name every place a secret value exists in plaintext - and the one place SOPS protects Respond to a pushed …

Chapter 05: CI/CD & Developer Guardrails

Explain what CI gives a team: the plan of a change for everyone to read, without tools or credentials on each laptop, and a place to discuss it before it runs Install the platform’s hooks, and explain why every …

Checkpoint A: Your Delivery Pipeline

This page is a consolidation — not new material. Its purpose is to show you the assembled delivery pipeline and the guardrails you can now rely on. What You Have Built You can now deliver code from a developer …

Chapter 06: Network Policies (Production Isolation)

Explain what NetworkPolicies protect: how far one compromised or misused pod can reach, and why a flat network makes every pod as dangerous as the most exposed one Read the platform’s policies and prove with a …

Chapter 07: Security Context & Pod Hardening

Explain what each setting of the platform’s security context takes away from a process that has been taken over - and what it does not Tell the layers apart: Pod Security at admission, the security context in the …

Chapter 08: Resource Management & QoS

Explain what a request and a limit each do - for the scheduler, the quota and the kernel - and why CPU and memory behave differently at the limit Name who stopped a container - the kernel’s OOM killer, the …

Chapter 09: Availability Engineering (HPA + PDB)

Explain who owns an app’s replica count - the HPA, not Git - and why a spec.replicas in Git would fight it during a peak Tell what a PodDisruptionBudget guards (evictions: a drain before an upgrade or a reboot) …

Checkpoint B: Your Runtime Safety Net

This page is a consolidation — not new material. Its purpose is to show how the four hardening chapters (06-09) work together to contain failure. What You Have Built Your workloads now run inside a layered safety net. …

Chapter 10: Version Promotion

Trace an image from its build to production - which environments are built, which are promoted - and prove by digest that production runs what staging tested Read an ImagePolicy: which tags it accepts, which one it …

Chapter 11: Observability (Metrics, Logs, Traces)

Explain what makes an alert worth sending - an action, an urgency, an owner - and remove the ones that are not Measure a service by what its users get, per environment, and alert on how fast the error budget burns …

Chapter 12: Backup & Restore

Explain why PostgreSQL, and why an operator (CloudNativePG) to run it on Kubernetes Tell a base backup from the WAL archive, and what each one gives a restore Restore a database to a point in time before a bad change - …

Chapter 13: Controlled Chaos

Run a chaos experiment as an experiment: steady state, hypothesis, one fault, measurement, comparison Measure what users get during a fault with an independent probe through the Service - not with kubectl get pods Say …

Chapter 14: AI-Assisted SRE Guardian

Explain how the Guardian turns signals - Alertmanager alerts, Kubernetes events, Flux stalls, its own scanners - into incidents: one per signal, its repeats counted rather than re-posted Prove its limits from the …

Chapter 15: 24/7 Production SRE

Declare a severity from what users get - and say why a critical alert is not automatically a Sev1 Run an incident with roles - even alone - and hand command over with an explicit confirmation Choose a mitigation on …

Chapter 16: Admission Policy Guardrails

Explain where an admission policy acts in the path of a request, and the difference between a policy that audits (records) and one that enforces (refuses) Promote a rule from Audit to Enforce for the namespaces where the …

Chapter 17: Supply Chain Security

Say what a keyless signature proves - which workflow, in which repository, on which branch signed this exact image - and what it does not: how the image was built is the provenance’s claim, what it contains is the …

Chapter 18: Rollback and Data Migrations

Explain why rolling back an image does not roll back a database, and why a point-in-time restore is not a rollback when users keep writing Split a schema change into expand and contract steps so that the previous release …

Chapter 19: Progressive Delivery

Explain what a canary limits - the share of requests a bad release gets, and for how long - and what it needs to decide: traffic, metrics, and a threshold Trace a release of the frontend through Flagger and Traefik, step …

Chapter 20: Platform Lifecycle

Say which parts of the platform may change by themselves and which only through a reviewed pull request - and why this platform lets nothing on its nodes move by itself Find what is out of date, and update it on your own …

Chapter 21: Credential & Access Lifecycle

Keep a registry of every credential the platform runs on - what it grants, every place a copy lives, when it was last replaced - and let a check turn the dates into a failure while there is still time Rotate a credential …

Chapter 22: Disaster Recovery

Say for each failure - a pod, a disk, a node, the cluster, the provider - what brings the platform back, how long that takes and how much data it can lose - and compare both with the objectives (RTO, RPO) Run a database …

Chapter 23: Capstone - Prove and Hand Over Your Platform

Take one change through the whole path on your own - pull request, staging, production - and prove it on the users’ path, not by a status Notice a breaking change you did not make, prove its cause from the …

Appendix: DNS and TLS Automation

Why This Appendix Exists The main course keeps early chapters focused on platform safety and GitOps. This appendix explains the edge automation layer used by the SafeOps platform: external-dns manages Cloudflare DNS …

Appendix: Local Development Environment

Why This Appendix Exists The main course teaches the production path first. This appendix shows the fastest safe feedback loop for local experimentation: a Terraform-managed kind cluster generated kubeconfig and context …

Intro: AI as a Very Well-Read Junior Engineer

More and more of those mistakes will not be typed by a person. AI already writes manifests, runs kubectl and opens pull requests. So the course also shows how to use AI in DevOps / SysOps / SRE work while keeping the …

Production-Grade Kubernetes with Guardrails & AI-Assisted SRE

Core Track (15 Chapters + Setup) Your Lab Environment Setup · ~1h fork of the platform repository, local kind cluster reconciled by Flux (local profile) your own SOPS key, the lab-pod.sh wrapper, the lab namespace the …