SafeOps Academy
Change, Protect and Recover a Kubernetes Platform - and Prove It
A practical course on one platform you run yourself: every change through Git, guardrails that hold whoever writes the change - you or your AI agent - and recovery you have tested. AI helps on the way, from writing changes to triaging alerts, and never skips a check. Every chapter starts from a real failure, and every claim ends with its evidence.
Watch the Intro
Five minutes that set the foundation: the SafeOps mental model, its working principles, and a preview of every chapter in the course.
Prerequisites
Who it is for: engineers who know Kubernetes basics - Pods, Deployments, Services, kubectl - and have run something real. The course is practice for the decisions production asks for; on its own it does not make anyone ready for any production.
You'll need a GitHub account, a few CLI tools (Docker, Terraform, kubectl, kind, flux, SOPS, age) and Docker with at least 4 CPUs and 8 GB of memory (12 GB recommended) for a two-node local cluster. No cloud account is required.
The whole core track runs on a free local kind cluster. A production-like Hetzner Cloud path exists as the optional Cloud track.
Core Track
Foundation-first path for platform, CI/CD, GitOps, observability, reliability, and on-call operations.
Chapter 00: Your Lab Environment
Build the local kind cluster once: fork, Flux local profile, your SOPS key, the baseline checks every chapter starts with.
CoreChapter 01: Blast Radius & the Shape of Safety
One correct command against the wrong cluster: the four rules and the layers that stop a mistake from spreading.
CoreChapter 02: Infrastructure as Code with Terraform
Terraform state as one shared, locked memory, and applying only the plan you have read - while it is fresh.
CoreChapter 03: GitOps with Flux
How Flux deploys from Git: reconciliation, prune, drift correction, and rollback with git revert.
CoreChapter 04: Secrets Management (SOPS + Age)
SOPS + age end-to-end: encrypt before commit, decrypt in-cluster with Flux.
CoreChapter 05: CI/CD & Developer Guardrails
Layered guardrails: pre-commit hooks, CI pipelines, approval gates, and AI-assisted review.
CoreChapter 06: Network Policies (Production Isolation)
Default-deny isolation and controlled allow rules with blocked-traffic debugging.
CoreChapter 07: Security Context & Pod Hardening
Pod hardening baseline: non-root, read-only rootfs, dropped caps, seccomp.
CoreChapter 08: Resource Management & QoS
Requests and limits as a budget: OOMKilled vs throttling vs eviction, QoS, quotas and the worst-case check.
CoreChapter 09: Availability Engineering (HPA + PDB)
HPA owns the replica count, PDBs guard evictions not rollouts, and every drain is simulated before it runs.
CoreChapter 10: Version Promotion
Production is a re-tag of the tested staging image, proven by digest; rollbacks pin the ImagePolicy.
CoreChapter 11: Observability (Metrics, Logs, Traces)
Alerts that ask for an action: user SLIs per environment, burn rates, grouped notifications by environment, and optionally traces to the database (Uptrace).
CoreChapter 12: Backup & Restore
Why PostgreSQL and CloudNativePG, base backups and the WAL archive, a point-in-time restore next to production, proven with data.
CoreChapter 13: Controlled Chaos
Chaos as an experiment: a written hypothesis, one fault, an outage measured through the Service, and a Chaos Monkey the cluster keeps on a leash.
CoreChapter 14: AI-Assisted SRE Guardian
An AI Guardian that turns alerts into incidents and proposes an analysis - read as a hypothesis, with limits the cluster enforces: read-only, no Secrets, a person decides.
CoreChapter 15: 24/7 Production SRE
One incident run end to end: severity from the impact, one commander, recovery proven on the users' path, and a postmortem that changes the system.
Advanced Track
Policy, supply-chain trust, progressive delivery, and rollback/data migration safety patterns.
Chapter 16: Admission Policy Guardrails
Admission policies that record everywhere and refuse where the evidence says it is safe, with exceptions only from Git.
AdvancedChapter 17: Supply Chain Security
Images signed by their own CI and verified at admission by the cluster's own verifier - and every signing change tested against real images.
AdvancedChapter 18: Rollback and Data Migrations
Schema changes the previous release can run on - expand, contract, roll forward - so a rollback is only an image and no user loses data.
AdvancedChapter 19: Progressive Delivery
A new frontend release reaches a share of the users first; its own metrics promote it or roll it back - and a canary that cannot see traffic is caught, not trusted.
AdvancedChapter 20: Platform Lifecycle
Keep the platform current on your own schedule: k3s and OS upgrades with drain and PDBs, the planned downtime one worker costs, and dependency updates.
AdvancedChapter 21: Credential & Access Lifecycle
Every credential in a registry with a date and a runbook; a rotation is done when the old value is refused; people get access by team, and the admin kubeconfig is break-glass.
AdvancedChapter 22: Disaster Recovery
Production's database as a primary and a replica, failover timed from the application's side; and the whole cluster lost and rebuilt - the platform from code, the data from its backup, declared in Git first.
AdvancedChapter 23: Capstone - Prove and Hand Over Your Platform
One change through staging and production with no commands to follow, a colleague's breaking change found and fixed through Git, an AI proposal judged - and the platform handed over, every claim with its evidence.