What is an AI Hallucination?

How AI hallucinations happen

A hallucination is a specific failure: the model asserts something false with confidence. A wrong answer flagged as uncertain, or an honest “I don’t know,” is not one. Hallucinations trace to three root causes. Training gaps: the model never learned a fact, or learned it wrong. Decoding randomness: generation is probabilistic, so even correct knowledge can sample into a wrong output. Context failures: the right information was available but the model could not use it, because the context was too long, badly structured, or fragmented.

The first two belong to the model vendor. The third is the one you control, and in production it is the most common. The scale of the problem is measurable even under ideal conditions: on Vectara’s leaderboard, models summarizing a document they were explicitly given still hallucinate at rates from 1.8% to over 23%, with reasoning-heavy models like o3-pro (23.3%) doing worse than simpler ones.

Why AI hallucinations matter

Hallucinations are the failure mode that forecloses correction. A model that knows it does not know can reach for a tool and retrieve the missing fact. A model certain it already knows never triggers that retrieval, so the error ships downstream looking exactly like a correct answer.

Context failures deserve the focus because they respond to engineering. Benchmarks comparing retrieval approaches found model faithfulness near-perfect in both conditions while correctness improved 25% from context quality alone: the model was faithfully reporting bad context, not inventing facts. And piling on more context backfires. A 172-billion-token study across 35 open models found fabrication rates roughly triple from 32K to 128K tokens, with every model exceeding 10% fabrication at 200K.

The mitigations that work are grounding and abstention. Ground answers in retrieved, relevant, well-structured context. Then treat “not found” as a valid outcome: Claude Opus 4.8 topped a hallucination benchmark with essentially unchanged accuracy, mainly by abstaining when uncertain instead of guessing.

Common misconceptions about AI hallucinations

• “Every wrong answer is a hallucination.” Most production errors are the model accurately reporting wrong or incomplete context. That is a retrieval problem with a different fix.
• “Better models will solve it.” Model progress helps faithfulness, but more capable models often hallucinate more on constrained tasks, and no model compensates for missing context.
• “More context means fewer hallucinations.” Past a point, added context dilutes attention and increases fabrication.
• “Prompting it not to hallucinate fixes it.” Instructions help at the margins but leave rates far too high for production; they cannot repair context quality.

AI hallucinations and Wire

Wire attacks the fixable cause: context failures. Containers hold structured, current entries that agents query through wire_search at answer time, so responses are grounded in retrieved records rather than the model’s recollection. When nothing matches, the tools return an honest empty result instead of the nearest wrong passage, giving the agent a clean signal to abstain or keep looking with wire_explore and wire_navigate.