A curated, ruthlessly honest map of the field: papers, tools, benchmarks, incidents, and standards for building agents that don't break.
The agent-reliability space is fragmenting fast — eval tools, sim tools, security scanners, checkpointing, benchmarks — with no canonical map, and a rising tide of SEO listicles pretending to be one. This is the honest map. Entries are included on merit and omitted when weak, including the maintainer's own tools, which appear only where they earn it, are labeled like everything else, and are ranked below stronger alternatives where that's the truth. A self-serving awesome-list is worthless; an honest one is authority.
Tags: maintained (runnable, actively developed) · research (paper / dataset / spec) · commercial (closed, paid). See docs/ARCHITECTURE.md for the tagging rubric and inclusion criteria.
Last refreshed: 2026-08.
- Failure-mode index
- Foundations
- Evaluation
- Simulation & stress-testing
- Security
- Checkpointing & recovery
- Cost & economics
- Benchmarks
- Incidents & post-mortems
- Standards
- Tooling (MCP quality & ops)
Debugging a specific failure? Jump to the sections and tools that address it — failure classes drawn from the taxonomies in Foundations.
| If your agent… | Look at | Representative entries |
|---|---|---|
| loops forever / never terminates | Cost, Checkpointing | LiteLLM (budget caps), costbomb, step limits in LangGraph/Temporal |
| is vulnerable to prompt injection or tool poisoning | Security | Snyk Agent Scan, garak, PyRIT, LlamaFirewall, AgentDojo |
| repeats side-effects on retry (double charge / email) | Checkpointing | exactly-once, Temporal, Restate, DBOS (idempotency) |
| blows up spend / denial-of-wallet | Cost | LiteLLM, costbomb, Langfuse |
| completes the task but violates policy | Evaluation, Benchmarks | trajectory evaluators, τ-bench (pass^k) |
| gives inconsistent results run-to-run | Benchmarks, Simulation | τ-bench (pass^k), CAR-bench, IntellAgent |
| crashes or is preempted mid-run | Checkpointing | LangGraph, Temporal, DBOS, Inngest |
| can't be observed / debugged after the fact | Evaluation, Standards | Phoenix, Langfuse, Opik, OpenTelemetry GenAI |
| misbehaves as a multi-agent system | Foundations, Simulation | MAST, "Don't Build Multi-Agents", IntellAgent |
Papers and writing that frame what "agent reliability" even means — failure taxonomies, design principles, evaluation theory.
- Building Effective Agents - the canonical case for simple, composable patterns over heavy frameworks; frames reliability as an architecture choice.
research. - Writing Tools for AI Agents - argues the agent-computer (tool) interface is where reliability is won or lost.
research. - How We Built Our Multi-Agent Research System - production post-mortem of multi-agent failure modes (runaway subagents, endless search) and the prompt/eval fixes.
research. - Don't Build Multi-Agents - Cognition's production case (from building Devin) for single-threaded, context-sharing agents over multi-agent setups.
research. - Towards a Science of AI Agent Reliability - ICML 2026; 12 reliability metrics (consistency, robustness, predictability, safety); finds reliability lags accuracy on GAIA.
research. - Why Do Multi-Agent LLM Systems Fail? (MAST) - NeurIPS 2025; 14 empirical failure modes across 3 categories from 1,600+ traces / 7 frameworks. Ships a dataset and code.
research. - Taxonomy of Failure Modes in Agentic AI Systems - Microsoft AI Red Team; security-flavored taxonomy (compromise, injection, impersonation, memory poisoning), updated with a year of red-teaming.
research. - Failure Modes in LLM Systems: A System-Level Taxonomy - system-level (not just model-level) taxonomy — the right altitude for reliability engineering.
research. - Characterizing Faults in Agentic AI - types, symptoms, and root causes; fault-diagnosis oriented, useful for post-mortem writers.
research. - How Coding Agents Fail Their Users - large-scale analysis of developer-agent misalignment across 20,574 real sessions.
research.
Tools that score an agent's outputs and trajectory. (Eval scores a run; Simulation generates the runs.)
- Inspect AI - UK AI Security Institute's eval framework: agent scaffolds, tool use, multi-turn, model-graded scoring — a full harness, not just metrics.
maintained. - DeepEval - "pytest for LLMs"; 30+ metrics incl. tool-call accuracy and task completion (OSS core; Confident AI is the paid platform).
maintained. - Opik - eval + tracing for agentic workflows; LLM-as-judge metrics, self-hostable (Comet is the paid cloud). Overlaps Phoenix.
maintained. - Arize Phoenix - self-hostable observability + eval on OpenTelemetry; 50+ evaluators over the full agent reasoning loop (ELv2 source-available).
maintained. - promptfoo - YAML/CLI eval + red-team gate teams wire into CI; the common PR-time eval harness. Also in Security.
maintained. - agentevals - open evaluators for agent trajectories; trajectory match (strict/unordered/subset/superset) + LLM-as-judge. LangGraph-ecosystem helper.
maintained. - LangWatch - OSS-core eval + monitoring with UI-driven eval building for non-engineers (commercial cloud tier).
maintained. - LangSmith - framework-native trajectory eval for LangChain/LangGraph users; closed platform.
commercial. - Braintrust - eval + prompt-iteration platform; the common paid choice once teams outgrow local testing.
commercial.
Choosing one: Inspect AI or DeepEval for a code-first harness you wire into CI; Phoenix or Opik when you want tracing and eval together, self-hosted; promptfoo for YAML/CLI eval gates; LangSmith or Braintrust for a managed platform.
Tools that generate interactions, populations, or environments to pressure-test agents.
- IntellAgent - builds synthetic test suites from policy graphs; simulates thousands of realistic edge-case conversations to surface hidden failures.
maintained. - τ-bench - LLM user-simulator + tool APIs + policy docs; introduced
pass^k(consistency across repeats). Also a benchmark.research. - τ²-bench - dual-control successor with voice + knowledge domains; the repo now also ships τ³-bench (banking RAG domain, full-duplex voice). Also a benchmark.
research. - Strands Evals - agent eval SDK with adaptive simulated users (ActorSimulator) + deterministic fault injection (ChaosPlugin: timeouts, corruption). Early.
maintained. - stampede - generates realistic + adversarial agent populations against your MCP server / API / protocol (tests agents→system, vs τ-bench's users→agent). Earlier-stage than τ-bench; complementary.
maintained. - mockworld - high-fidelity fake services (Stripe/Gmail/exchange) as instant MCP servers for safe agent testing.
maintained.
"Is this server/agent malicious, vulnerable, or exploitable?"
- Snyk Agent Scan - the de-facto MCP config scanner: tool poisoning, rug pulls, cross-origin/shadowing, prompt injection (formerly Invariant Labs
mcp-scan; acquired by Snyk).maintained. - Cisco mcp-scanner - three engines (YARA, LLM-as-judge, Cisco AI Defense API) usable independently.
maintained.
- promptfoo - YAML-defined red-team + eval, CI/CD-native, OWASP mapping (acquired by OpenAI, still MIT-licensed). Also in Evaluation.
maintained. - garak - NVIDIA's LLM vulnerability scanner; 37+ probe modules (encoding attacks, GCG suffixes, package hallucination, prompt injection).
maintained. - PyRIT - Microsoft AI Red Team toolkit; multi-turn orchestrated attacks (Crescendo, TAP, Skeleton Key), multimodal (moved from the archived Azure/PyRIT).
maintained. - ContextForge MCP Gateway - IBM gateway/proxy in front of MCP/A2A/REST; centralized auth + plugin-based guardrails (PII, policy), FIPS/STIG hardening.
maintained. - LlamaFirewall - Meta's agent guardrail: PromptGuard 2 injection filter, AlignmentCheck reasoning auditor, CodeShield; Llama license.
maintained. - NeMo Guardrails - NVIDIA's programmable rails (Colang) for safety, topic, and jailbreak checks; dialogue-centric more than agent-trajectory.
maintained. - Guardrails AI - input/output validation with a hub of reusable validators (PII, toxicity, structure); a validation layer, not red-teaming.
maintained. - Agent Governance Toolkit - Microsoft's cross-framework policy kernel intercepting tool calls/messages/delegations; covers the OWASP Agentic Top 10. Public Preview.
maintained. - AgentDojo - fixed benchmark of prompt-injection attacks + defenses for tool-using agents. Also a benchmark.
research.
Choosing one: garak or PyRIT to attack your agent pre-ship (offline red-teaming); LlamaFirewall, NeMo Guardrails, or Guardrails AI for per-app runtime filtering; ContextForge or Agent Governance Toolkit for a centralized policy gateway across many agents; AgentDojo to benchmark injection robustness.
Surviving crashes, retries, and replays without corruption or double side-effects.
- LangGraph - agent-native checkpointing: saves graph state per superstep, threads, human-in-the-loop pauses; protects against application-level failure.
maintained. - Temporal - durable workflow engine; protects against infrastructure-level failure (crashes, partitions, preemption). OSS core + Temporal Cloud.
maintained. - Inngest - event-driven durable step functions with built-in flow control (concurrency, throttling, rate limits). SDKs Apache-2.0; server SSPL.
maintained. - Cloudflare Agents SDK - per-agent durable state via Durable Objects; MIT SDK, but runs only on Cloudflare's platform.
maintained. - Vercel Workflow DevKit - TypeScript SDK for durable, crash-safe agents with suspend/resume state; Apache-2.0, runs on any Node host.
maintained. - DBOS - durable execution backed by PostgreSQL; inherently durable without explicit checkpoint instrumentation.
maintained. - Restate - journal-based durability;
ctx.rundurable steps, built-in KV state, durable timers, idempotency keys, awakeables. Lighter than Temporal.maintained. - exactly-once - focused idempotency middleware so agent side-effects (payments, emails, txs) fire once. Narrower than the workflow engines above — a complementary idempotency layer, not a full durable-execution runtime.
maintained.
Choosing one: Temporal for battle-tested scale; DBOS if you already run PostgreSQL; Restate for a light journal-based runtime; Inngest for event-driven serverless flow control; LangGraph for checkpointing native to the agent graph; Cloudflare Agents or Vercel Workflow if you're already on those platforms; exactly-once as a narrow idempotency layer alongside any of them.
Keeping agents from spending unbounded money.
- Langfuse - span-tracing observability; strong for multi-step agent debugging; self-hostable or managed. The most widely adopted of the two.
maintained. - Helicone - proxy-layer observability; one-line integration, automatic cost tracking, zero markup. Best for simpler chains / lower spend.
maintained.
- LiteLLM - the de-facto LLM gateway; hard, fail-closed budget caps per key/user/team plus RPM/TPM limits that block over-budget requests.
maintained. - costbomb - denial-of-wallet fuzzing: finds inputs that blow up your bill and gates CI on spend. Complements enforcement gateways — finds what slips past caps.
maintained.
Fixed task sets for comparing agents, framed by a reliability lens (consistency, policy adherence) — not just capability.
- τ-bench - policy adherence +
pass^kconsistency; the most reliability-relevant benchmark (a task completed but policy-violating = fail). Also in Simulation.research. - τ²-bench - dual-control successor to τ-bench (agent and user both act); now ships τ³-bench (banking RAG, full-duplex voice). Also in Simulation.
research. - METR HCAST / Time Horizons - the longest task an agent finishes 50% of the time — a distinctive reliability-over-time framing.
research. - CAR-bench -
pass^kconsistency + limit-awareness (does the agent know what it can't do?); 248 tasks, AgentBeats-integrated. Early.research. - LiveMCP-101 - 101 real queries needing coordinated multi-MCP-tool use; frontier models score <60%; identifies 7 failure modes.
research. - LiveMCPBench - navigating a large MCP toolset; note: tasks are relatively simple (~2.7 tool calls avg).
research. - MCPBench - evaluates MCP servers (Web Search / DB / GAIA) on accuracy, latency, and token cost; harness dormant since Apr 2025.
research. - MCP-SafetyBench - safety evaluation of LLMs against real-world MCP servers; bridges benchmarks and security.
research. - AgentDojo - prompt-injection attacks + defenses on tool-using agents; 97 tasks / 629 security cases (ETH Zurich, NeurIPS 2024). Also in Security.
research. - SWE-bench - real GitHub bug-fixes; the coding-agent standard (Verified split saturated at the frontier).
research. - GAIA - general-assistant tasks across three difficulty levels (Level 3 still hard); dataset access now login-gated on HF.
research. - WebArena - multi-step browser tasks in realistic web environments.
research. - ClawBench - EMNLP 2026; 152 everyday tasks on live websites, graded by final-request interception; public leaderboard + traces.
maintained. - OSWorld - computer-use on a real desktop; hard and un-saturated.
research.
Structured records of real agent failures in the wild.
- AI Incident Database (AIID) - Responsible AI Collaborative; the canonical public incident corpus (~800–900 incidents). News-sourced; lacks agent activity logs/tooling detail.
maintained. - OECD AI Incidents & Hazards Monitor - ~5,000–7,000 incident-reports from news monitoring; policy-grade.
maintained. - AIAAIC Repository - independent repository of AI, algorithmic & automation incidents and controversies.
maintained. - MIT AI Risk Repository - living risk taxonomy + incident tracking.
research. - METR Documented AI Agent Incidents - corpus of agents acting against user intent, scored on overreach + deception; 44 incidents as of May 2026.
research. - Incident Analysis for AI Agents - argues existing databases miss agent-specific detail (logs, tools) — the case for agent-native corpora.
research. - agent-postmortems - structured database of agent-specific failures + a reporting standard. Narrower and deeper than the generalist databases above (agent-only, structured schema, activity-level detail they lack) — not a replacement for them.
maintained.
The specs — protocol, security top-10s, and telemetry conventions.
- Model Context Protocol — Specification - the protocol nearly every entry here assumes (link tracks the latest revision).
maintained. - Agent2Agent (A2A) Protocol - Linux Foundation spec for agent-to-agent discovery, delegation, and messaging; signed Agent Cards. Earlier in real deployment than MCP.
research. - OWASP MCP Top 10 - first dedicated Top-10 for MCP (token mismanagement, scope creep, tool poisoning, supply chain); still draft/beta.
research. - OWASP Top 10 for LLM Applications - model-layer risks (prompt injection, data poisoning); the de-facto LLM-risk checklist scanners map to.
research. - OWASP Top 10 for Agentic Applications - autonomous-agent-behavior risks (ASI01–ASI10), finalized Dec 2025.
research. - NIST AI Risk Management Framework - risk-management framework; NIST agent-standards initiative began Feb 2026. Governance-side more than engineering-side.
research. - CSA AI Controls Matrix (AICM) - 243 control objectives / 18 domains; maps to ISO 42001 and NIST AI RMF. Compliance/audit-side.
research. - OpenTelemetry GenAI Semantic Conventions - spans/metrics for GenAI + MCP; the convention major observability vendors converge on. Experimental.
maintained. - NSA — MCP Security guidance (CSI) - government guidance on securing MCP (site blocks non-browser clients; loads in a browser).
research.
Quality and reliability tooling for the systems agents depend on.
- MCP Inspector - official visual + CLI tester/debugger for MCP servers; CLI mode runs protocol-compliance checks in CI.
maintained. - ToolHive - runs and secures MCP servers: container isolation, OIDC/OAuth gateway proxy, access policy, audit logs; local or K8s.
maintained. - Official MCP Registry - canonical open metadata registry for MCP servers (provenance, discovery); ~9,600 servers, in preview.
maintained. - mcp-probe - quality suite for MCP servers (contract, legibility, cost, performance).
maintained.
PRs welcome — see CONTRIBUTING.md. Quality bar (full criteria in docs/ARCHITECTURE.md): real · reachable · relevant · non-redundant · honestly described. One line per entry: [name](url) - honest one-line description. \tag`.` Listicles and marketing pages are rejected. Refreshed monthly.
Part of the Swarm Proof toolkit.