Log — All posts

devopskubernetesreliability

How I deploy software without downtime — or 2am pages

July 14, 2026 · Northlight Studio

Most businesses don't think about how their software gets deployed — until a release takes the site down during business hours, and suddenly everyone cares. The good news: this is a solved problem. Here's the approach I build, explained without assuming you run infrastructure for a living.

The old way, and why it hurts

For years, deploying meant someone logging into a server and running commands by hand. It works right up until it doesn't: a step gets skipped, two people change things at once, and nobody's quite sure what's actually running in production versus what's in the code. When something breaks, you're reconstructing history under pressure.

The idea: Git is the source of truth

The pipeline I build is based on GitOps. The principle is simple: the desired state of your entire system lives in Git — the same version-control system that already tracks your code. Want to know what's running in production? Look at the repository. Want to change it? Make a change in Git. Want to undo a bad release? Revert the change, exactly like undoing any other edit.

Nothing is done by hand on a server. That single rule eliminates a whole category of "how did it get like this?" problems.

How a change actually reaches customers

Here's the journey, start to finish:

  1. Build & test (CI). When code is ready, an automated pipeline builds it into a container — a sealed, reproducible package — and runs the test suite. If tests fail, it stops here. Nothing broken gets further.
  2. Record the new version in Git (CD). The pipeline updates a configuration file that says "production should now run version 1.4.2." That's a normal Git commit — reviewable, timestamped, reversible.
  3. The cluster reconciles itself. A tool called Flux continuously watches that Git repository. The moment it sees the new version, it updates the running system to match — automatically. No human runs the deploy.
  4. Rollout with health checks. New copies start alongside the old ones. Only once the new version passes its health checks does traffic shift over. If it never gets healthy, the old version keeps serving. Customers see nothing.

That last point is the "no downtime" part. There's never a moment where the app is off. The new version has to prove it's healthy before it's trusted with real traffic.

A stressed engineer kneeling in front of a server rack, cable in hand, with a confused thought bubble and a red warning alert — the manual-deploy panic that GitOps replaces

Multiple environments, same pipeline

Real projects need staging environments to test in before touching production. The same setup handles dev, testing, and production from one source of truth — each one just points at a different folder in Git. A change gets proven in staging, then promoted to production by the same mechanism, so there are no surprises from an untested path.

Why this matters to a business

  • Fewer outages. Bad releases can't get healthy, so they can't take you down.
  • Instant rollback. A one-line revert in Git puts the previous version back in minutes.
  • A complete audit trail. Every change to production is a Git commit — who, what, when.
  • No key-person risk. The system deploys itself; it isn't locked in one person's head.

You don't need to understand any of the tooling to benefit from it. You just get releases that ship on a Tuesday afternoon without anyone holding their breath — which is exactly how it should be.