Most AI agent demos show the happy path. A request arrives, the agent finds the right information, updates the right system, and sends the right response.
Production is less polite.
The CRM times out after the agent sends the email. The model returns an answer, but the workflow loses the transaction ID. A connector retries a step that already completed. The approval queue is unavailable. A vendor changes an API response. Now the team has to answer a harder question: what happened, what completed, and what is safe to do next?
If your implementation plan says "retry on failure," it is not finished. A retry can recover a temporary read error. It can also create a duplicate order, message, payment request, or case when the first attempt succeeded but the confirmation never came back.
Design failure recovery before production access. The workflow should detect trouble, stop the right actions, preserve evidence, route work to a person, and restart without guessing.
Define failure in business terms
A technical error is only one kind of failure. The workflow can return HTTP 200 and still do the wrong job.
Start with one transaction. List the outcomes that the business cannot accept: a missed request, duplicate action, wrong record update, unauthorized message, stale decision, partial write, or item stuck without an owner. Then identify which system signals could reveal each outcome.
This matters because different failures need different responses. A model timeout before any action may be safe to retry. An uncertain write to a customer system should pause until the team confirms whether the first action completed. A low-confidence recommendation may belong in human review rather than an automated retry loop.
If the workflow itself is still vague, use the AI agent workflow readiness test first. Recovery rules depend on a clear trigger, expected result, systems involved, and consequence of being wrong.
Track the transaction, not just the model call
The model call is one step inside a business transaction. Recovery needs a durable record that follows the work from trigger through final outcome.
Assign each transaction a unique ID. Record the workflow version, model or policy version, input reference, tool calls, approvals, attempted writes, system responses, timestamps, and final state. Avoid treating a final model response as proof that the work completed.
The AI agent data map helps document each read, write, retained copy, and recipient. Add state to that map. For every step, define what "not started," "in progress," "confirmed," "failed," and "unknown" mean.
Unknown deserves its own state. If an API request timed out after submission, the system may not know whether the downstream write happened. Marking that item failed and immediately trying again is how duplicates appear.
Make retries safe before turning them on
A retry policy needs more than a count and a delay.
Decide which operations are safe to repeat. Reads usually carry less business risk than writes. A write should use an idempotency key, unique transaction reference, duplicate check, or downstream constraint when the system supports one. When it does not, the workflow needs a confirmation query or human review before repeating the action.
Set a backoff schedule and a hard retry limit. Endless retries can turn a small outage into a large backlog or a burst of actions when the service returns. Route exhausted retries to a visible exception queue with the transaction context intact.
Test ugly timing. Let the downstream action complete, then cut the connection before the acknowledgment returns. Delay the response past the timeout. Return a malformed payload. Send the same event twice. These cases tell you whether the design prevents duplicate business actions, not merely whether the code catches an exception.
Plan for partial completion
Multi-step workflows rarely fail cleanly at the beginning.
An agent may create a CRM task, fail to send the message, and never update the reporting table. Another may reserve inventory, fail during payment review, and leave the reservation open. Rolling back every completed step is not always possible or desirable.
For each write, choose one recovery treatment:
| Treatment | When it fits | What must be documented |
|---|---|---|
| Retry | The action is safe to repeat and the first result can be identified | Retry limit, delay, duplicate control, success check |
| Compensate | A completed action can be reversed through a separate business action | Reversal owner, side effects, evidence, approval |
| Reconcile | The workflow must compare records across systems before deciding | Source of truth, matching rule, exception threshold |
| Escalate | The state is uncertain or the consequence is too high for automation | Queue, owner, response target, context packet |
| Accept and continue | The failed step is optional and the business accepts the degraded result | Allowed condition, user notice, follow-up work |
A compensation step is not the same as deleting evidence. If the workflow reverses an action, keep the original attempt, the reversal, the reason, and the person or policy that authorized it.
Build a manual fallback that people can use
"The team will handle it manually" sounds reasonable until the outage starts and nobody knows which items completed.
A usable fallback needs an intake point, owner, current transaction state, source records, last confirmed action, next safe action, and a way to prevent the automated workflow from processing the same item while a person works it. The team also needs access to the systems and data required to finish the job.
Do not copy every prompt and log line into the handoff. Give the reviewer the evidence needed to decide: original request, validated data, completed actions, uncertain actions, error, customer impact, and permitted next steps.
Set a capacity limit. If the manual team can process 40 exceptions a day and the automated workflow can create 400, the fallback will collapse during a real outage. Throttle intake, prioritize by impact, or stop the workflow before the queue becomes unrecoverable.
Separate pause, shutdown, and recovery
Stopping new work does not fix transactions already in motion.
Define three controls. Pause prevents new transactions from starting. Shutdown revokes or disables the agent's ability to act. Recovery deals with items that were processing when the stop occurred. These controls may have different owners and technical steps.
The NIST Generative AI Profile recommends protocols that allow generative AI systems to be deactivated when necessary. It also calls for incident response plans for third-party AI technologies, regular rehearsals, retrospective improvement, and tested rollover or fallback processes that may include manual processing.
That is the right standard for a production workflow. The controls should exist, the people should know how to use them, and the team should rehearse them before an incident.
Do not restart into the same failure
Teams often focus on stopping an agent and give less thought to turning it back on.
Set restart criteria before launch. Confirm the cause is understood or contained. Reconcile in-flight transactions. Clear or quarantine bad inputs. Verify credentials, connectors, queues, and monitoring. Run a small controlled batch. Compare the results with the source systems before restoring full volume.
Decide what happens to the backlog. Replaying everything in order may send stale messages or apply decisions that no longer make sense. Each queued item needs an age limit and a revalidation rule.
Preserve the failure record after service returns. NIST recommends after-action reviews for generative AI incident response and updates to the response process when gaps appear. A review should produce specific changes to tests, thresholds, runbooks, monitoring, or workflow design.
Use a failure-recovery acceptance test
Before production, run a recovery exercise against representative test data. Do not tell the operator exactly where the failure will occur.
The test should prove that the team can detect the problem, identify affected transactions, distinguish confirmed from uncertain actions, stop new work, prevent duplicates, route exceptions, continue critical work manually, reconcile systems, and restart with a controlled batch.
Measure detection time, time to safe pause, number of uncertain transactions, duplicate actions, lost records, exception backlog, manual throughput, recovery time, and reconciliation errors. Add those results to the AI agent production scorecard. Reliability claims should come from observed recovery behavior, not a vendor's architecture slide.
NIST's profile says an AI system should be able to fail safely and that safety metrics should reflect reliability, real-time monitoring, and response times for AI system failures. That does not mean every low-risk workflow needs a heavy incident program. It means the recovery design should match the consequence of failure.
Set the production decision
Approve production only when the team can answer five questions with evidence:
Which failures are safe to retry? How do we prevent duplicate writes? What happens when completion is uncertain? Who owns the manual queue? What must be true before restart?
If those answers live only in the developer's head, the workflow is not ready. If the vendor says the platform "handles retries," ask to see the transaction states, duplicate controls, exception records, shutdown method, and recovery test.
A working happy path proves the agent can do the job. A tested fallback proves the business can live with it when the job goes wrong.