The third time my phone buzzed at 2 AM in a single month, I stopped calling it bad luck. This is a story about automated disk space monitoring, the kind that catches a disk before it fills up at night instead of waking you once it already has. For years I was the automation. I logged in half-asleep, deleted a few gigabytes of old logs, closed the ticket, and crawled back into bed knowing the exact same page would fire again next week. It always did.

After that third night I was angry enough to actually fix it.

The reactive loop nobody wants to admit to

The old pattern is embarrassingly common. A disk crosses 100% at 2 AM. An alert fires. Someone, usually whoever is on-call and least awake, scrambles to free space, resolves the ticket, and goes back to sleep. No trend. No forecast. Nobody noticed that the same disk had been climbing a percent a day for three weeks and was always going to blow on a Wednesday. Just reactive firefighting, on repeat.

The estate made it worse. We were watching over a hundred production disks across roughly fifty servers, a mixed Windows and Linux fleet. Windows and Linux report disk usage differently, so a single naive query lies to you about half the time. On top of that, NFS mounts kept tripping false positives: a "full" network share that had nothing to do with the host it was mounted on. I had personally burned hours chasing ghosts that were never real.

Flip the whole thing: report before the incident, not after

The fix was not another alert. It was inverting the timeline.

I built an Azure Logic App that runs every morning at six, before anyone wakes up and before the first coffee. It fires KQL queries against Log Analytics across all hundred-plus disks, with Windows and Linux on separate query paths because they simply do not report the same way. NFS mounts get filtered out automatically, so the ghosts never make it into the results. By the time the team opens their laptops, the report is already sitting in their inbox.

Two lanes, because not every number is an emergency

The real trick was routing. One threshold for everything is how you either drown in noise or sleep through the fire. So the system runs two lanes.

  • Above 90% is critical. It goes straight to on-call with a high-priority ticket and a full escalation chain. This is a genuine emergency and it should wake someone up.
  • Above 75% is proactive. It lands quietly in the daily trending report. The team sees it over morning coffee, plans the cleanup, and fixes it Tuesday afternoon instead of scrambling at 2 AM on Wednesday.

Context matters more than any single percentage. Eighty percent on a 2 TB volume is a shrug. Eighty percent on a 50 GB disk is a countdown. The trending report is what turns a raw number into a decision you can make with your eyes open and your coffee in hand.

No cron job to babysit

I deliberately did not put a cron job on a VM. Cron jobs on VMs are a lie you tell yourself. They work until someone patches the box, reboots it, or lets a certificate expire, and then your monitoring is quietly dead at exactly the moment you need it.

This is serverless. You pay per execution. It has self-healing retries on transient failures. It has now run for months without a single manual intervention: no babysitting, no "is the monitor still alive?" checks, no monitor-for-the-monitor.

The best incident is the one that never happens

That last line sounds like a slogan, but it is the entire return on investment. Months without a single 2 AM page for disk space. The disks still fill up, because that is physics, but now they announce it politely at 6 AM while there is still room and still time to act.

If you run anything at scale, the lesson generalizes past disks. Build the system once so you can sleep, instead of being the system every night. Reactive monitoring makes you the retry loop. Proactive monitoring makes the machine do the boring part and only wakes you for the things that genuinely cannot wait.

When was the last time you got paged for something a script could have prevented?