First week at a new banking client, I asked what I thought was a warm-up question: "Where are the health checks?" The room went a little quiet. It turned out the Azure App Service health checks were missing everywhere — twenty-six App Services running in production, processing real transactions, serving real customers, and not a single one had a health check endpoint configured. In a bank. The platform was simply assuming everything was healthy.
Hope is not a strategy.
The difference between "running" and "working"
Here is the thing most dashboards get wrong. There is a world of difference between "the server is up" and "the application actually works." An App Service instance can be running, the process alive, the CPU graph looking perfectly calm — and the app inside can be throwing 500s at every single request. Without a health probe, nothing notices. The load balancer keeps happily routing traffic to a dead instance. Users keep hitting errors. And in a bank, an error is not a blank page — it is a failed transaction, a payment that did not go through, a customer staring at their screen wondering what just happened.
These are zombie instances: up by every crude measure, dead by the only one that matters. The worst part is how you find out. Not from a graph, not from an alert — from a customer picking up the phone. By the time a human is manually checking what went wrong, the damage is already counted in real money and real trust.
Why I didn't just flip a switch
The fix sounds trivial: configure a health check endpoint, done. In a banking environment nothing is that simple, and it shouldn't be. You do not yolo infrastructure changes across production payment systems. So I rolled it out in five rounds instead of one big bang.
- Round 1 — the lowest-traffic, least-critical services first, purely to validate the pattern.
- Round 2 — internal APIs, where a mistake stays inside the building.
- Round 3 — mid-tier services.
- Round 4 — customer-facing applications.
- Round 5 — and only then, the critical banking workloads.
Every single round went through the Change Advisory Board with a full rollback plan documented before I touched anything. That is not bureaucracy for its own sake — it is what lets you move confidently in an environment where the blast radius includes people's money. By the time I reached the critical workloads, the pattern had already proven itself four times over.
Policy, not a wiki page
Rolling out the probes fixed the present. It did nothing for the future. Six months later, someone spins up App Service number twenty-seven, forgets the health check, and the whole blind spot is back. Documentation does not solve this. A "best practices" wiki page is where good intentions go to be forgotten.
So the part I am proudest of is not the rollout — it is what came after. I set up an Azure Policy that enforces health check configuration at deployment time. An actual, enforced gate — not a suggestion. Deploy a new App Service without a health check and the deployment gets flagged automatically. On top of that, automatic unhealthy-instance replacement now kicks in the moment a probe fails: the platform pulls the bad instance and spins up a fresh one. The team finds out from the system, not from a customer complaint.
The result that lasts
Twenty-six services. Five rounds. Zero downtime, zero incidents. But the number I actually care about is the one you can't fit on a slide: from that point on, every new App Service is born healthy. Not because an engineer remembered to configure a probe — because the system will not let them forget. The knowledge stopped living in one person's head and moved into the platform itself.
That is the real lesson from those twenty-six services. Health checks are not a nice-to-have you get around to. They are the difference between finding out from your monitoring and finding out from your customers. And the only health check that reliably exists is the one a policy refuses to deploy without.
So I will leave you with the same question I asked in that room during my first week: how many of your production services have a real health check right now — and what enforces it when the next person forgets?