For most of my career, DevOps meant writing YAML. Somewhere around 2026 I stopped, and I started teaching infrastructure to think instead. That sentence sounds like a slide from a vendor booth, so let me be precise about what I mean: I set out to build an AI-powered DevOps platform, real infrastructure that detects anomalies, predicts failures, and acts on them in production. Not a conference demo. Not a robot on a slide. Something running against real customers and real money.

This is the honest version of how it went, including the part nobody puts in the keynote.

From reactive firefighting to predictive

The old loop is familiar to anyone who has carried a pager. An anomaly appears, an alert fires, a human investigates, a human decides, a human fixes. By the time you are awake enough to read the dashboard, the incident has already cost you.

The loop I wanted looked different. An anomaly appears, the system correlates it against historical patterns, it proposes a fix, and a human approves. When confidence is high enough, it fixes itself and logs everything for review. The goal was never to remove people. It was to stop waking them up for problems the system already understands.

The stack

I kept the stack deliberately boring where boring was safe, and interesting only where it earned it.

  • Python for the intelligence layer: the correlation logic, the model orchestration, the decision engine.
  • Terraform for the infrastructure itself, so every change the platform proposes is a plan a human can read.
  • Azure OpenAI for the reasoning: pattern matching across logs and metrics that no person could hold in their head.

Terraform mattered more than it looks. Because every action lands as a Terraform plan, an AI proposal is not a black box mutating your cloud. It is a diff. You can read it, reject it, or let it run. That single design choice is what made the whole thing defensible in a review meeting.

Trust but verify

The principle the entire platform is built on is trust but verify. I split actions by blast radius.

Low-risk work executes on its own: tagging untagged resources, scheduling non-production environments to sleep overnight, right-sizing a dev database nobody is watching. Anything that touches production networking or scaling goes through a human approval gate. Every decision, automatic or gated, is logged and auditable. If someone asks why the platform did something at 3am, there is an answer, with a timestamp and a reason.

Cost optimization was the first thing that paid for itself

I expected anomaly detection to be the star. Instead the cost layer won the room first.

The AI went through more than 200 resources and found money we had stopped seeing. Idle VMs running 24/7 for no reason anyone could remember. Oversized instances that could drop a tier without a single user noticing. Workloads that only needed business hours but were billed around the clock. Forgotten snapshots quietly accruing charges. It surfaced roughly $40K a month in waste on the first pass.

Here is the part I find genuinely interesting: humans miss this because we get used to the status quo. The instance was always that size. The VM was always on. Familiarity blinds you. The model has no such bias, and it trains on your specific infrastructure patterns, not generic benchmarks pulled from a whitepaper.

The demo that changed the conversation

Getting the technology right was hard. Getting stakeholders to believe that infrastructure making its own decisions is not science fiction took roughly twice as long.

What finally landed was not a chart. It was a replay. The previous quarter, a memory leak had cost the team four hours of diagnosis before we found and patched it. I fed the same conditions to the platform. It identified the pattern in 11 seconds and proposed the exact fix we had eventually shipped by hand.

Nobody argued after that.

What actually changed

A few months in, the numbers that mattered were not about the AI. They were about the people.

  • Mean time to detection dropped from minutes to seconds.
  • False positive rate fell about 60%, the number that actually saves your on-call from alert fatigue.
  • Night pages were cut in half.
  • The engineering team started sleeping through the night.

That last one is not a metric you put on a dashboard, but it is the one I care about most.

Where I draw the line

I do not believe in fully autonomous production. I believe in a system that handles the boring, obvious, well-understood work on its own and hands the ambiguous, expensive, irreversible decisions to a human with full context. The line between auto-fix and human-in-the-loop is not fixed. It moves as the system earns trust, one audited decision at a time.

We do not need more tools. We have plenty of tools. We need systems that learn from the infrastructure they manage, and the discipline to let them prove it before we hand over the keys.