Every Monday morning, the same thing. Someone was supposed to trigger the weekly data import. Again, they didn't. The data team is waiting, the product team is annoyed, and the one person who actually knows how to run it is on vacation with the laptop switched off — exactly where they should be. This is the story of how I decided to automate a weekly data import that had quietly become a forgotten task, held together by one human's memory — a cron job that never was, running instead on whether someone remembered to click a button at 6 in the morning.
The pattern nobody names
Let me describe the setup, because you probably have one just like it somewhere.
A critical data import ran once a week. It was triggered manually by a single person. The credentials lived somewhere between a shared doc and their own head. There was no monitoring — if the import failed, nobody knew until hours later, when someone downstream finally complained that the numbers looked stale.
The failure mode was always the same:
- The responsible person forgot, or overslept, or was out.
- The import silently didn't run.
- The data team scrambled, the product team got angry.
- The "fix" was to remind that person to run it earlier next week.
That last part is the tell. When your remediation for an outage is "remind a human to be more reliable," you don't have a people problem. You have an engineering gap you keep choosing not to close.
The week it finally broke in the open, the responsible person had gone on leave. Nobody else knew the credentials. Nobody else knew the steps. The data team waited until 11 AM before anyone even asked, "did the import actually run?" It had not.
What I actually built
The whole thing took less than a day to replace, and none of it is exotic. That's the uncomfortable part — this was always solvable.
- A scheduled Azure trigger fires every Monday at 6 AM, before anyone is awake. No human in the loop, no button to click.
- Bash scripts orchestrate the run — they pull the source, validate that the imported data actually looks right, and retry automatically on transient failures instead of dying on the first hiccup.
- Credentials moved into Key Vault instead of living in someone's notebook. The scripts read them at runtime; nobody stores them anywhere they can rot or leak.
- Slack alerting on exhausted retries. If the job genuinely can't recover, the team gets pinged instantly — so they know before standup, not after a stakeholder does.
- Every single run is logged and tracked. Success or failure, there's an audit trail. When someone asks "did Monday's import run?", the answer is a dashboard, not a shrug.
Notice what each piece removes. The schedule removes the need to remember. The validation removes silent bad data. The retries remove the fragility. Key Vault removes the single-person credential dependency. The alerting removes the hours-long detection gap. The logging removes the "did it even run?" question entirely.
The result
Since the automation went live: zero missed imports. Not one. The data is ready before standup every single week. Nobody needs to remember, nobody needs to be awake, nobody needs to be reachable.
And here's the part nobody expected. The person who used to be the single point of failure — the one who couldn't fully switch off, who ran "just in case" Slack checks on their days off — took a real vacation for the first time in months. Left the laptop at home. Because the system genuinely didn't need them anymore.
That, to me, is the whole point. Automation isn't only about uptime metrics. It's about not turning a teammate into a cron job with legs.
The real lesson
If a human has to remember to trigger something every week, it will eventually be forgotten. Not because they're careless — because they're human, and humans go on leave, change jobs, or just have a bad Monday. Betting your data pipeline on nobody ever having a bad Monday is not a strategy.
So here's the question worth sitting with. Somewhere on your team, right now, one person holds all the context for a process nobody else understands. Picture them handing in their notice tomorrow.
Feeling a little nervous? Good. That's the task to automate first.