AI Agents Need Recoverability Before Autonomy

The current discussion about AI agents focuses too much on what they can do without human involvement. I think that starts at the wrong end of the problem.

Before I let an agent act independently, I want to know what happens when it makes a reasonable but wrong decision. Can we see what it changed? Can we understand why? Can we undo the action without creating a second problem?

If the answer is no, the agent is not ready for autonomy. It is merely capable of producing work quickly.

This distinction matters because speed changes the cost of mistakes. A person may update ten customer records incorrectly before someone notices. An agent can apply the same faulty interpretation across an entire segment before the first complaint reaches the team. The individual mistake is not necessarily worse, but its distribution is.

Product and growth teams already work with systems where small decisions have commercial consequences. A pricing rule can affect margin. A campaign configuration can spend money. A CRM update can change which customers receive an offer. A support action can issue a refund or close a case. Giving an agent access to these systems is not comparable to asking it to summarise meeting notes.

That is why task completion is a weak measure of agent quality. An agent can complete a task exactly as instructed and still damage the outcome. It may optimise campaign volume while attracting low-quality demand. It may resolve support tickets faster by choosing answers that reduce follow-up rather than solve the customer’s problem. It may clean up a pipeline by removing records that look incomplete but still represent valuable opportunities.

I would start with recoverability instead. Every action should have a clear owner, a visible history and a defined boundary. The team should know which changes can be automatically reversed, which require approval and which should never be delegated.

This is not an argument for putting a human approval step in front of everything. That usually creates the appearance of control while people learn to click approve without checking. The better approach is to match control to the consequence.

An agent drafting a campaign variation can work with broad freedom because publication remains separate. An agent changing bids can operate within an agreed range and budget. An agent issuing refunds may be limited by value, customer status or reason. An agent changing prices across a product portfolio should probably remain a recommendation system until the organization can inspect and reverse its decisions reliably.

I would also measure how often the team needs to intervene, but not treat every intervention as failure. Early interventions reveal where instructions, data or system boundaries are unclear. The useful question is whether the same category of mistake keeps returning. If it does, adding another prompt is rarely enough. The product itself needs a better constraint or a clearer decision rule.

This has implications for roadmaps. Teams are tempted to plan agent work around capabilities: connect another system, support another task, remove another approval. I would plan it around increasing levels of consequence. Begin with actions that are observable and cheap to reverse. Then expand access only when the evidence shows that errors are understood and contained.

The aim is not to build an agent that never gets things wrong. We do not hold people or software to that standard either. The aim is to build a product where mistakes remain manageable.

Autonomy is attractive because it promises less manual work. But autonomy without recoverability simply moves the work. Instead of performing the task, people spend their time investigating unexpected changes and repairing trust with customers or colleagues.

For me, the test is simple: if I cannot explain how we recover from a bad action, I am not ready to automate the action. The agent may be impressive, but the product is unfinished.