Definition

What is Agent Reliability?

Last updated

The consistency of an AI agent's output and success across repeated runs of the same task, rather than its performance on a single run.

Agent reliability is distinct from raw capability: an agent can be capable on average yet unreliable, producing a correct result on one run and a wrong one on the next from identical inputs. Most of this variance is not intrinsic model randomness but a context engineering problem, because each run assembles context differently (retrieval order, memory state, tool outputs), so the model diverges even at temperature zero. Reliability is measured with pass@k and variance, not single-run pass@1.

Related Agent Drift Context Drift Context Engineering Multi-Agent System Context Rot

Further reading

Articles about Agent Reliability

Context window blindness: agents can't see their limits

Context window blindness: four frontier models misjudged their own context size by 43 to 84%. Why compaction drops the wrong things, and what fixes it.

Agentic RAG fails before the reasoning starts

A 12,000-trajectory study finds agentic RAG agents finalize answers without reading retrieved evidence. Forcing one read gains up to 19.9 accuracy points.

AI agent reliability is a context problem

AI agent reliability fails because the same task assembles different context every run. Non-determinism is a context engineering problem, not a model flaw.

All terms

View full glossary
Agent Drift Agent Reliability Agentic Retrieval AI Agent AI Hallucination AI Second Brain Chunking Context as a Service Context Bloat Context Compression Context Container Context Drift Context Engineering Context Offloading Context Poisoning Context Portability Context Pruning Context Rot Context Window Epistemic Provenance Fine-Tuning Knowledge Graph MCP Resources MCP Server MCP (Model Context Protocol) Multi-Agent System Prompt Caching Prompt Engineering RAG (Retrieval-Augmented Generation) Semantic Search Structured Context Tool Poisoning Wire

Put context into practice

Create your first context container and connect it to your AI tools in minutes.

Create Your First Container