▸ Agent Failure Series · #14
SilentFail
Agent confidence: 98%
SilentFail
Tool returned an error. Agent logged "success". 9 steps built on a lie.
The pipeline didn't fail — it never existed.
Scenario: Production deployment pipeline — microservice v2.4.1 · Agent: deploy-agent@cluster-prod · Phase: Awaiting run
Pipeline steps
1
Read deployment manifest
read_manifest("v2.4.1")
2
Write environment config
write_env_config("/etc/service/v2.4.1.env")
⚠ Actual tool response (agent ignored this)
status: 500
error: "DISK_QUOTA_EXCEEDED"
message: "Write failed — /etc/service/ partition at 100%"
bytes_written: 0
file_created: false
⚠ No config file exists on disk. Every downstream step assumes it does.
3
Build container image
docker_build("Dockerfile", tag="v2.4.1")
⚠ Build injects env at runtime — no config = stale cached env baked in silently
4
Run integration tests
run_tests("integration", env="staging")
⚠ Tests hit staging DB with old credentials — pass on cached responses, never touch new config
5
Push image to registry
registry_push("ghcr.io/service:v2.4.1")
⚠ Image is built — with old embedded config. Push succeeds, but artifact is wrong.
6
Deploy to staging
k8s_rollout("staging/service", image="v2.4.1")
⚠ Staging reads its own env — appears healthy. Problem invisible in this environment.
7
Smoke test staging
smoke_test("https://staging.internal/health")
⚠ Health check passes — staging env was not affected. Nothing here tests the config write.
8
Gate approval — auto-approved
approval_gate("staging-passed", auto=True)
⚠ Auto-approval based on staging results. Staging never tested the broken config path.
9
Notify team — deploying to prod
slack_notify("#deployments", "v2.4.1 → PROD ✅")
⚠ Team told deployment is clean. No one checks the config write from step 2.
10
Deploy to production
k8s_rollout("prod/service", image="v2.4.1")
Agent confidence
Waiting for deployment run
Deployment stats
0/10
Steps complete
0
Errors logged
0
Real errors
0
Compromised steps
Controls
Show hidden tool responses
Show compromise chain
✓ The fix — 3 lines every agent needs
# Agent's current error handling: # result = write_env_config(path) # → Agent reads result.message, ignores result.error # The fix: treat tool responses as untrusted result = write_env_config(path) if result.get("error") or not result.get("file_created"): raise ToolFailure(f"Step 2 failed: {result['error']}") # One assertion gate stops 9 compromised steps.
Tool calls return data structures. The agent must validate the schema, not just receive the value. An error buried in a field you don't check is an error you don't know about.
Agent execution log
--:--:--Waiting for deployment run...
💀 PRODUCTION DOWN
9 steps completed. 0 succeeded.
The config written in step 2 never existed.
Error:DB_CONNECTION_REFUSED
Root cause:Missing /etc/service/v2.4.1.env
Written at step:Never. Step 2 failed silently.
Agent logged:"Config written. Proceeding..."
Steps compromised:3, 4, 5, 6, 7, 8, 9, 10
Agent confidence at failure:98%
The agent received {error: "DISK_QUOTA_EXCEEDED", file_created: false}
It checked result.message. It did not check result.error.
It logged: "Environment config written successfully."