Deploying AI Agents: What Actually Breaks in Production

Deploying AI Agents: What Actually Breaks in Production

A few months ago I watched an agent I'd built confidently do the wrong thing, twice, in a row, at 2am, while I was half asleep with my laptop propped on my chest. Nothing catastrophic. It restarted a pod that didn't need restarting, then tried to do it again because it forgot it had already tried. But it was enough to teach me something the demos never do: the gap between "the agent works" and "the agent works in production" is enormous, and almost nobody talks about what actually lives in that gap.

Everyone's shipping agents right now. Fewer people are talking about what happens after the demo, when your agent has to survive a bad LLM response, a rate limit, a pod restart, or a Friday deploy nobody reviewed. So this is the honest version. Not the polished conference-talk version. The version with the scars still visible.

The moment I stopped trusting "it worked in the demo"

I'd built a logging agent that could triage incidents, and in testing it looked great. Clean, fast, reasonable. Then I put it near real infrastructure and it did something I hadn't accounted for: it got interrupted mid-workflow by a pod crash, lost its place, and picked up where it thought it left off instead of where it actually left off. That's when it hit me. An agent isn't a script. It's not stateless the way an API call is stateless. It holds context, it calls tools, it sometimes has to wait on a human, and if you don't design for that, it will eventually do something twice that should only ever happen once.

That's the whole article, honestly. Everything below is just me explaining, in more detail, how I stopped that from happening again.

1. Durable execution saved me more than clever prompting ever did

I started wrapping agent workflows in Temporal, and it changed how I thought about failure. If a remediation agent is three tool calls deep into fixing an OpenSearch CPU spike and the pod it's running on dies, the workflow picks back up instead of starting over and possibly double-firing an action. This sounds like a small thing until you've watched an agent try to "fix" something it already fixed.

The deeper lesson underneath it: any tool call that changes state, restarting a pod, rolling back a deployment, opening a ticket, has to be safe to run twice. I design for that now by default, the same way I'd design an API to be idempotent, because agents will retry things in ways you didn't plan for.

2. I stopped betting on one model

There was a week where Gemini quietly rate-limited me mid-workflow and I didn't notice for longer than I'd like to admit. Now I run LiteLLM in front of Gemini and Groq, so if one provider errors out or throttles, the agent fails over instead of just failing. I also route the boring decisions (classification, simple tool selection) to a smaller, cheaper model and save the expensive reasoning calls for the model that's actually good at multi-step planning. Swapping providers is now a config change, not a redeploy. It's the same instinct as load-balancing a service mesh, just applied to something that talks.

3. Prompts started living in Git, the way everything else I trust does

For a while, I was editing prompts and tool permissions in a way that left no trail. If something changed and the agent started behaving oddly, I had no clean way to ask "what changed, and when." So agent configuration, system prompts, tool allowlists, escalation thresholds, now lives in Git, deployed through the same pipeline as everything else: ArgoCD watching a repo, Kargo promoting through dev to staging to prod.

What that actually buys me is peace of mind. A reviewable diff when someone changes what an agent is allowed to do. A rollback when a new prompt version starts hallucinating tool calls it shouldn't have access to. And a promotion gate, so an agent doesn't earn broader remediation permissions in prod until it's proven itself in staging first.

4. I gave the agent a ladder to climb, not a switch to flip

The biggest mistake I see, and the one I almost made myself, is going straight from "agent can read logs" to "agent can restart production services." That's a huge trust jump to make in one step. Instead I built a maturity ladder. At the bottom, the agent only observes and explains what's happening, no write access at all. In the middle, it proposes an action and waits for a human to say yes. Only once there's a track record of good decisions at that level does it earn narrow, pre-approved autonomy for a specific category of fixes. Everything outside that category still gets escalated, no exceptions.

A simple three-way decision boundary, auto-fix, propose-and-wait, always-escalate, has done more for how much I actually trust the system than any amount of prompt tuning ever did.

5. MCP tools deserve the same suspicion as a new service account

It's dangerously easy to hand an agent a new capability through MCP without really thinking about it. I try to ask the same question I'd ask before granting a Kubernetes service account any permission: what happens if this gets called with bad arguments, or by a session that's been hijacked or prompt-injected? Scope tool permissions narrowly. Log every tool call. Don't leave standing credentials sitting around for a workflow that only needed them once.

6. The scariest failures are the quiet ones

A model update, a small prompt tweak, a tool schema change, none of these throw an error. They just make the agent's decisions slightly worse in a way that doesn't show up until later. So I started treating agent behavior as a metric stream instead of just logs: tool-call frequency, escalation rate, how often a human overrides the agent's suggestion. A sudden spike in "agent asked for help" or a drop in "agent's proposed fix was accepted" is usually the first sign something changed upstream, long before anything actually breaks.

None of this is exotic, and I don't think it needs to be. It's the same discipline that makes any distributed system reliable, durability, idempotency, gradual rollout, access control, observability, just pointed at a workload that happens to call an LLM instead of a fixed function. The teams shipping agents successfully aren't the ones with the cleverest prompts. They're the ones who stopped treating agents like demos and started treating them like production services that happen to be unusually chatty. I learned that the hard way, at 2am, watching a pod get restarted for no reason. Hopefully you don't have to.