When a tool call fails: validation, error-as-feedback, and escalation

In a demo, every tool call returns exactly the clean data the happy path expects. In production, an upstream API times out, returns an empty list, hands back JSON that doesn't match its own docs, or — worst of all — returns something well-formed but wrong. An agent is only as reliable as its flakiest tool, so failure handling at the tool boundary is a large part of what separates a demo from a system. The tool-calling-done-right post covers defining tools safely; this is about surviving what they return.

Validate the result, not just the request

Most people schema-validate the tool call the model produces. Fewer validate the tool's response before the agent acts on it — and that's where the nastier failures live. Parse and schema-check every result at the boundary (a Zod/Pydantic model per tool), so a flight API that suddenly returns price as a string, or omits a field it used to send, is caught here instead of cascading into a confidently wrong final answer three steps later. The structured-outputs post makes the same argument for the model's own output; apply it to everything crossing a boundary.

Classify the failure — each kind wants a different response

• Transient transport errors (timeout, 5xx, rate limit) — retry with backoff, capped. These usually clear on their own.
• Empty-but-valid results (no flights for that date) — not an error; a fact the agent must reason about, not retry into.
• Schema-invalid results (wrong shape, missing field) — do not blindly retry; a deterministic bad response will fail identically. Surface it or escalate.
• Valid-but-wrong results (plausible data, subtly incorrect) — invisible to error handling entirely; only evals and cross-checks catch these.

Error-as-feedback: hand the failure back to the model

The single most useful pattern: when a tool fails recoverably, don't crash the run — feed the error back into the loop as an observation. The agent's reason–act–observe cycle (the agent-loops post) already has a slot for this; a failed observation is still an observation. 'The events API returned no results for 2026-04-31' lets the model notice the date is invalid and correct itself; 'the pricing tool rejected currency=US$, expected an ISO code' lets it retry with 'USD'. Given a clear, specific error, a capable model recovers from a surprising share of failures on its own — far more gracefully than any exception handler you'd write.

Bound it, though: cap the number of self-correction attempts so a tool that always fails can't spin the loop forever. After N tries, stop and fall back.

Retry, then escalate

Layer the fallbacks. Retry transient failures a couple of times with backoff. If a cheaper model keeps emitting malformed tool calls, escalate to a stronger one — the same retry-then-escalate ladder as the cost-routing post, applied to tool reliability rather than answer quality (a small model often fumbles a strict tool schema a larger one gets right). And when the tool genuinely can't answer, degrade deterministically — a cached value, a sensible default, or an honest 'I couldn't retrieve that' — rather than letting the model paper over the gap with an invention.

How you know a tool failed in production

All of the above is moot if failures are invisible. A tool error doesn't throw a stack trace you'll see — the code ran fine; the data was just bad — so you have to instrument it. Emit a structured event per tool call (tool name, arguments, latency, outcome, retry count) and trace the whole agent run end-to-end, so 'the bot was weird' becomes 'the pricing tool returned empty on step 3' (the LangSmith-observability post is exactly this). Then alert on the rates: a climbing tool-error rate or retry rate is your early warning that an upstream API changed under you.

The one failure this won't catch is the valid-but-wrong result — it never registers as an error. That's the silent-failure problem from the NLQ post, and the only defense is evaluation against known-good cases, not monitoring. Wire failure metrics and traces for the loud failures; lean on evals for the quiet ones.

Treat every tool result as hostile until validated. The loud failures you retry or escalate; the quiet, plausible-but-wrong ones you catch with evals — because your error logs never will.