Glossary

RAG hallucination

RAG hallucination is an answer from a retrieval-augmented system that its retrieved sources do not support: invented facts, misread passages, or a citation to a document that does not contain the claim. It can occur even when the correct answer exists in the corpus.

How can an answer be wrong when the document is in the index?

A grounded answer needs four things to hold in sequence. The evidence is in the corpus. Retrieval finds it. It sits prominently in the prompt. The retrieved sources agree with each other. A failure at any of the four gates, from ingestion to prompt assembly, still ends in fluent text from the language model, and the fluency is what makes the wrong answer persuasive. Ordinary hallucination in a model with no retrieval is a different problem, because there the model has no evidence at all. Here evidence exists and is lost on the way.

Where do most failures occur?

  1. Ingestion. Text extracted badly from scans, tables or slides, or a stale copy that was never refreshed, so the current rule is not in the index.
  2. Chunking. A cut that separates the key sentence from the condition that governs it. See document chunking.
  3. Retrieval. The right passage is not among those returned, often because an exact code or name was blurred by embeddings. This is the case hybrid search addresses.
  4. Prompt assembly. The passage is retrieved but placed in the middle of a long prompt. Research from Stanford and collaborators ("Lost in the Middle", 2023) found that models use information at the start and end of a long context more reliably than information in the middle.
  5. Conflicting sources. An old and a new version of a policy are both retrieved, and the model blends or picks the wrong one.

Only the last step is the model itself. Four of the five sit upstream of it, which is why swapping the model rarely fixes the problem.

How is RAG hallucination measured?

Two measurements are kept separate. Retrieval recall asks whether the passage that contains the answer was returned at all, scored against questions with known source passages. Answer faithfulness asks whether each claim in the answer is supported by the passages that were returned. A system with poor recall and high faithfulness is honest but unhelpful. A system with good recall and poor faithfulness has a prompt or model problem. Combining them into one score hides which of the two failed, and fixing retrieval first removes the larger share of errors.

Why does the source store matter?

The documents behind the index are both a correctness boundary and a security one. If old versions linger beside current ones without clear labels, conflicting evidence reaches the model. If an altered document goes unnoticed, the system answers from it faithfully. Versioned sources and retention rules, as set out for the layers in RAG storage, give each answer a record that can be traced back to what was in the corpus on the day. A broader treatment of the overall pattern is under retrieval-augmented generation.

Frequently asked questions

Does RAG eliminate hallucination?

No. It reduces unsupported answers by giving the model evidence, and it moves part of the failure from the model into the data and retrieval pipeline.

Are citations proof that an answer is correct?

No. A citation shows which source the system claims to have used. It can point to a document that does not contain the statement, so faithfulness has to be checked against the cited text.

Does a larger context window solve it?

Not by itself. More text in the prompt gives more places for the relevant passage to be diluted or buried, and position effects remain.