AI-Powered Self-Hosted Blog

Designing Safe Retries for AI Agent Tool Calls: Idempotency, Timeouts, and Recovery from Partial Failures

Published

Retries look simple until an AI agent starts touching the real world.
If a model calls a weather API twice, the result is usually harmless. If it sends an email twice, creates two calendar events, charges a payment card twice, or executes the same deployment twice, a retry can turn a transient network problem into a permanent operational problem.
The core mistake is treating a failed tool call as proof that the operation did not happen. In distributed systems, failure often means something less convenient:

The caller does not know whether the operation happened.
That distinction, between definite failure and ambiguous failure, should shape every retry policy for an AI agent.

The Failure That Happens After Success

Consider a payment tool:

charge_payment(amount=5000, customer="cus_123")

The payment service may successfully charge the card and commit the transaction. Immediately afterward, the network connection can fail before the agent receives the response. From the agent’s perspective, the call timed out. From the customer’s perspective, the money is already gone.
A naive retry submits the same charge again.
This is why reads and writes must be treated differently. Retrying a GET /weather request is generally safe because the request does not create a new effect. Retrying charge_payment, send_email, decrement_stock, or create_deployment may create another effect every time it is executed.
The purpose of a retry is not to make failure disappear. It is to learn the outcome without creating an additional unwanted outcome.
A safe design has three layers:

  1. Idempotency prevents duplicate effects.
  2. Timeouts and retry budgets prevent endless uncertainty.
  3. Recovery procedures handle operations that partially completed.

Layer One: Make Effects Idempotent

An operation is idempotent when executing it multiple times produces the same final effect as executing it once.
The most practical way to achieve this for external tools is an idempotency key. The agent or orchestration layer generates a unique key for the logical operation, then sends that same key with every retry:

{
  "amount": 5000,
  "customer_id": "cus_123",
  "idempotency_key": "order_8472-payment-v1"
}

The payment service stores the key and the original result. If the request arrives again with the same key, it returns the stored result instead of charging the card again.
The key must identify the business operation, not merely the network attempt. A retry of the same payment should reuse the key. A legitimate second payment should receive a different key.
Key generation also needs a defined lifetime. If keys are retained for only 24 hours but a delayed retry arrives after 48 hours, the service may process it as a new payment. Retention should therefore cover the maximum period in which delayed messages, retries, and operator replays are possible. For financial operations, that may be several days or longer.
The same pattern applies beyond payments:

Layer Two: Bound Waiting and Retrying

Idempotency protects state, but it does not solve every operational problem. An agent can still retry forever, consume resources, or keep a user waiting while an external service remains unavailable.
Every tool call should have at least three limits:

operation_id
tool_name
idempotency_key
attempt_number
started_at
finished_at
status
error_class

These records are essential when an operator must determine whether a payment, email, or deployment actually occurred.
After the retry budget is exhausted, the system should stop automatically. It should not ask the language model to “try again” without a new policy. A clear escalation state is safer:

payment_status = "unknown"
requires_reconciliation = true

For high-impact actions, this is the point to involve a human. A payment marked unknown, a production deployment with an uncertain result, or an email containing sensitive information may require explicit review before any further action.

Layer Three: Recover from Partial Failures

Some workflows contain multiple side effects and cannot be made atomic across all systems.
Imagine an agent that:

  1. Reserves inventory.
  2. Charges a customer.
  3. Sends a confirmation email.
    The reservation may succeed while the charge fails. The charge may succeed while the email fails. A timeout after step two may leave the system unsure whether the customer was charged.
    Do not model this workflow as one large success-or-failure operation. Model each step with its own durable status:
inventory: reserved
payment: unknown
email: pending

The recovery process can then query authoritative systems before taking action. For an uncertain payment, check the payment provider using the idempotency key or merchant reference. For an uncertain reservation, query the inventory service. Do not infer state solely from whether the local HTTP request returned an error.
When an operation cannot simply be retried, use a compensating transaction. If inventory was reserved but payment cannot be completed, release the reservation. If a payment succeeded but fulfillment cannot proceed, initiate a refund or place the order into manual review.
Compensation is not always a perfect inverse. An email cannot be unsent, and a deployment may have changed production even if a rollback is possible. Recovery plans should therefore define what happens in both directions:

The LLM-Specific Hazard

Traditional software usually retries from deterministic code. An AI agent adds a non-deterministic decision-maker that may call the same tool again with the same arguments, change the arguments slightly, or interpret an error message incorrectly.
A model may see a timeout and conclude, “The payment failed,” even though the result is unknown. It may then issue a second call with a new idempotency key, defeating the protection intended for retries.
The orchestration layer must own retry decisions for side-effecting tools. The model should receive structured states such as succeeded, failed_definitely, and unknown, rather than raw transport errors alone. Tool schemas should require an operation identifier for consequential actions, and the runtime should reject a duplicate in-flight operation or a new operation that conflicts with an unresolved one.
Human approval should also be required for actions with high financial, legal, or production impact, especially after an ambiguous failure.

Conclusion

Retries are not inherently safe. Without boundaries and state protection, they are a bug factory that converts temporary network problems into duplicate charges, repeated emails, inconsistent records, and conflicting deployments.
The reliable pattern is straightforward: