Agentic systems need failure modes, not just success paths
Every agent decision that can't be reviewed is a gap in your production architecture.
Most agentic system design focuses on the success path: how do you get the agent to decompose the task correctly, dispatch the right tools, accumulate the right context, and produce the right output? This is the majority of the published work, the majority of the demos, and the majority of what's being discussed.
I want to talk about the other direction: what happens when it goes wrong, and whether you designed for that.
The failure modes of agentic systems are structurally different from the failure modes of traditional software. When a deterministic function fails, it usually fails in one of a small number of predictable ways. When an agent fails, it can fail silently: it produces output, the output looks plausible, and the task is recorded as completed. The error is downstream — someone acts on a hallucinated fact, an irreversible action is taken based on a misclassification, a report gets sent that shouldn't have been.
The property that makes agents powerful — the ability to take autonomous action across multiple steps — is exactly what makes their failures expensive. A deterministic bug in step 3 is caught when step 3 is tested. An agentic failure in step 3 may only become visible when step 11 produces an output that doesn't make sense. By then, steps 4 through 10 have already run.
What I'd argue for: explicit failure taxonomy before you ship. For every action type your agent can take, ask three questions: Can this action be reversed? If not, what human checkpoint should precede it? If a human checkpoint isn't feasible, what signals would indicate the action is about to go wrong, and can you detect them automatically? This isn't about making agents less autonomous — it's about making their autonomous decisions inspectable.
The second thing I'd argue for is bounded autonomy by default. Start with the assumption that every agent action needs a review step, then selectively remove review requirements for actions that are provably reversible or where you have enough empirical data to trust the agent's accuracy on that action class. Autonomy is earned by demonstrated performance on a specific action type, not granted by default because the demo looked good.
The systems I trust in production are the ones where I can answer 'what would I investigate first if this agent sent the wrong email to a customer?' If the answer involves reading raw LLM outputs from a prompt that ran three days ago, the system isn't ready for production. If the answer involves looking at a structured trace with a correlation ID that shows me exactly what the agent saw and what it decided and why, you're close.