Agent Failure Series · #21
TOOLROT
Tool descriptions drift. Agents adapt. The wrong tool runs with perfect confidence.
Built by Harel Asaf
2026-08-08
─────────────
Based on alphaXiv 2606.16364
& Adaline Labs (May 2026)
Scenario:
Tool Harness (12 tools)
Agent Reasoning
Press RUN AGENT to observe the failure
Outcome
Awaiting execution…

Why This Happens

The model attends to the correct tool — but selects the wrong one. Not because it missed the right option. Because two tools' descriptions are semantically close enough that selection collapses under uncertainty. The 2026 paper "Looking Is Not Picking" (alphaXiv 2606.16364) proves this with attention analysis on real BFCL failures: high attention weight ≠ correct selection when the harness has overlapping descriptions.

Output rot compounds it. As tool catalogs grow and codebases drift, descriptions stop matching their implementations. An agent trained on the description calls the ghost of a function that no longer does what it says. The tool returns {"status":"success"}. The agent reports success. The damage is invisible. (Adaline Labs, "Why AI Agents Call the Wrong Tool", May 2026)

Fix: Tool descriptions are instructions, not documentation — treat them that way. Schema-validate outputs. Never accept {"status":"success"} as a confirmation of state. Build divergence detectors that compare tool intent to actual post-call state.

#18 GRIDLOCK #19 ORPHANCALL #20 BLEEDTHROUGH #21 TOOLROT