First week on a new project. I opened the architecture diagram expecting boxes and lines, and instead I got arrows. Everywhere. Forty-seven microservices, and it looked like every single one had been wired directly to every other one. No Azure API Management gateway, no front door, nothing — just a web of direct calls, pure microservices chaos, that nobody could trace, nobody could throttle, and nobody could secure.

So I asked the team the simplest question I could think of: how do you debug a failed request? The answer told me everything I needed to know. "We grep the logs across six services and hope the timestamps line up."

That was the moment I understood what I was looking at. This was not an architecture. It was a chain reaction waiting for a reason to start.

The Tuesday it actually exploded

It did not wait long. One service started misbehaving and hammered another one with retries. That service buckled, which triggered more retries, which dragged three more services down the chain. A cascade, in production, on a Tuesday afternoon — because Tuesdays are apparently when things explode.

Here is what made it so painful. There was no auth between services, so anything could call anything. There were no rate limits, so one bad actor could flood the whole system. And there was no tracing, so once things went sideways, the only tool anyone had was "grep and hope." That is not observability. That is a prayer with a command line.

One gateway in front of everything

The fix was not exotic. It was discipline. I put Azure API Management in front of the whole mess, so every API call goes through the gateway instead of straight into another service's side door.

At the edge, the gateway now enforces the things the services had been quietly skipping:

  • OAuth and JWT validation at the entry point, so services stop pretending they can trust each other by default.
  • Mutual TLS between the gateway and the backends.
  • Rate limiting per client, per subscription, per product — so one misbehaving caller can never take down the whole chain again.
  • Full request tracing from entry point to backend and back, so a failed request is something you follow, not something you guess at.

The team went from "we grep six service logs and hope" to "we open the gateway dashboard and look." That shift alone changed how the on-call weeks felt.

Policies as code, not clicks in a portal

A gateway you configure by hand in a portal is just a new place to make undocumented changes. So none of this lives in the portal. All of it lives in Terraform.

The APIM instance itself is Terraform-managed. API definitions are imported straight from the OpenAPI specs the services already publish. The XML policy templates — rate limits, auth, rewrites — are version-controlled and peer-reviewed like any other code, because a policy that decides who can call your API deserves at least as much scrutiny as a function that formats a date.

Promotion runs through one pipeline: dev to staging to production, the same definitions each time, zero drift. Nobody "quickly fixes something in prod" and forgets to tell the rest of the world. If it is not in Git, it does not exist.

The gateway cannot become the new single point of failure

There is an obvious trap here. You spend three months making the gateway the front door for forty-seven services, and now the gateway is the one thing that can take all of them down at once. You have not removed the single point of failure — you have relocated it and put a nicer logo on it.

So the deployment is multi-region and active-active. Updates go out blue-green, so a bad rollout swaps back instead of taking the front door with it. The gateway that was supposed to eliminate the single point of failure is not allowed to quietly become one.

Build it right, build it in code, sleep at night

The whole transformation took three months — from spaghetti to a single, observable, code-managed gateway. It was not glamorous work. A lot of it was reading OpenAPI specs, arguing about policy scopes, and testing failover at hours no one enjoys.

But here is the honest test of an architecture: can you trace a failure without opening six terminals, and can one bad service ruin everyone else's afternoon? If the answers are "yes" and "no," you can actually sleep.

If your microservices are talking to each other directly, with no gateway between them, you do not have an architecture yet. You have a time bomb with good uptime — for now.

Ever traced a production failure across five services with nothing but raw logs and optimism? I have. Once was enough.