Glossary

What Is RAG Hallucination?

RAG hallucination is when a retrieval-augmented generation system produces an answer its retrieved sources do not support — inventing a fact, misreading a passage, or attributing a claim to a document that does not contain it. The retrieval step exists to prevent exactly this, which is what makes the failure both surprising and, in an enterprise setting, expensive.

The term is practitioner shorthand rather than a term of art. The research literature separates factuality — does the output match the world? — from faithfulness — does the output match the source it was given? A plain language-model hallucination is usually a factuality failure. A RAG hallucination is usually a faithfulness failure: the evidence was supplied, or should have been, and the answer departed from it anyway.

The nuance that matters for anyone operating one of these systems is that RAG hallucination is a pipeline failure, not only a model failure. Grounding can be lost anywhere between a document arriving in storage and an answer reaching a user, and most of those places sit upstream of the model.

Why retrieval reduces hallucination without eliminating it

Retrieval changes what the model conditions on. It does not change the model's disposition to produce a fluent, plausible continuation. Supplying evidence helps when that evidence is present in the corpus, actually retrieved, salient in the prompt, and internally consistent — and the architecture guarantees none of those four things. It also introduces an entire retrieval pipeline that can fail on its own terms.

The measured picture is less tidy than the marketing. The figure most often quoted — a reduction of "over 60%" — comes from a 2021 study of knowledge-grounded dialogue in which the no-retrieval baseline hallucinated in roughly 68 percent of responses. It is good evidence of direction and useless as a number to quote for a current system. There is no credible, generalisable reduction figure for modern frontier models; the percentages circulating in "AI hallucination statistics" articles have no traceable methodology behind them.

What recent peer-reviewed work does establish is the shape of what remains. Joren et al. (ICLR 2025) built a classifier for whether retrieved context was sufficient to answer a question at all, and found that large models handle sufficient context well but, when context is insufficient, tend to answer anyway rather than abstain. Smaller models hallucinated even when the context was sufficient. Their purpose-built selective-generation method improved the share of correct answers among those attempted by 2 to 10 percent — a real gain, and nowhere near elimination.

Where grounding actually gets lost

Barnett et al. (CAIN 2024) catalogued seven failure points from three production RAG systems; a 2026 taxonomy extends the list to 33 across the pipeline and grades its own evidence base, which is a useful corrective to confident lists. The failures that account for most ungrounded output:

The distribution matters more than the list. The first five are data and retrieval problems; only the last three are model problems. Teams routinely respond to hallucination by changing model or prompt when their losses are occurring several stages earlier, where they are cheaper to fix and easier to measure.

One item that appears on many published lists deserves a caveat: summarisation quietly dropping qualifiers and hedges is a plausible mechanism with no measured support behind it — a fair description of a good deal of RAG advice.

Two opposite failures: ignoring the context, and trusting it too much

Most discussion of RAG hallucination assumes the model ignored what it was given. That does happen, and it has a name. The knowledge-conflict survey by Xu et al. (EMNLP 2024) distinguishes context-memory conflict — retrieved evidence disagreeing with what the model already believes — from inter-context conflict, where the retrieved passages disagree with each other.

The mirror image gets less attention and matters more for enterprise corpora. ClashEval tested frontier models against deliberately perturbed retrieved content and found they abandoned their own correct prior knowledge more than 60 percent of the time when a retrieved passage contradicted it — more readily when their prior was weakly held, less readily as the perturbation became more absurd. It is a preprint, so the figure is indicative rather than settled. The direction is the uncomfortable part: a RAG system is not only capable of ignoring good context, it is also disposed to believe bad context.

Which reframes what the index is. If the model will generally defer to what it retrieves, then what sits in the corpus is a correctness question, not merely a coverage question.

Position and context length still matter

Liu et al., in the paper that gave the effect its name — "Lost in the Middle", published in TACL in 2024 — found accuracy highest when the relevant passage sat near the beginning or end of the context and materially lower when the model had to reach into the middle, including on models built specifically for long contexts. The finding has held, with nuance: Byerly and Khashabi (TACL 2026) ran 651 experiments across seven datasets and eight models, confirmed the positional patterns, and measured accuracy drops of 20 to 25 percent for middle and end positions in 30-document contexts. The precise curve is model-dependent and not always the symmetric U-shape of the original study.

Two consequences follow. The order in which retrieved passages are packed into the prompt is a variable worth tuning, not an implementation detail. And the argument that a long enough context window removes the need to retrieve well does not survive contact with evidence: NoLiMa (ICML 2025) tested twelve models advertising 128K to 1M token contexts and found ten of them below half their short-context baseline at 32K — GPT-4o falling from 99.3 to 69.7 percent — because models lean on literal lexical overlap that disappears once question and evidence are worded differently.

One widely offered remedy makes this worse. Byerly and Khashabi found that sampling the model repeatedly and aggregating — self-consistency — amplified position bias rather than correcting it, because the errors across samples are correlated rather than independent.

How groundedness gets measured

Measurement is the step most deployments skip, which is why RAG hallucination is usually discovered by users rather than by engineering.

The standard automated approach decomposes a response into individual claims and scores the fraction the retrieved context supports. Ragas (Es et al., EACL 2024) implements this as its faithfulness metric. A naming caution for anyone reading older guidance: several Ragas metrics have been renamed — "answer relevancy" is now Response Relevancy and "answer correctness" is now Factual Correctness — and a great deal of published advice still uses the old names.

Verification does not require a frontier model. MiniCheck (Tang et al., EMNLP 2024) showed a 770-million-parameter entailment checker matching GPT-4 accuracy on grounding checks at roughly one four-hundredth of the cost, which is what makes checking every answer in production economically possible rather than aspirational.

Citations deserve a specific warning, because they are the most common answer to "how do we stop it hallucinating". Emitting a citation is not the same as being supported by one. The ALCE benchmark (Gao et al., EMNLP 2023) found that even the strongest systems failed to fully support their own citations roughly half the time on long-form questions. Citations are an auditability mechanism — they let a human check a claim quickly. They are not a prevention mechanism, and treating them as one produces a system that is wrong in a more credible-looking way.

What helps, and what is folklore

The interventions with real evidence behind them are mostly unglamorous:

• Hybrid retrieval. Lexical (BM25-style) and dense vector search combined beat either alone in the EMNLP 2024 best-practices study — and on some document types, financial reporting among them, plain BM25 still beats state-of-the-art dense retrieval.
• Reranking. Anthropic's published work on contextual retrieval reports retrieval failure falling from 5.7 to 1.9 percent by combining contextual embeddings, contextual BM25 and a reranking stage. A vendor result rather than peer review, but the method is disclosed — and reranking buys accuracy with latency.
• Chunk size. The same study measured roughly 97 percent faithfulness at 256-token chunks against roughly 80 percent at 2,048. One of the largest levers available, and the least discussed.
• Packing order. Most relevant passages nearest the query — the practical application of the position-bias finding above.
• Deciding whether to retrieve at all. Classifying queries so retrieval is skipped where it will not help improved both answer quality and latency.
• Groundedness checking on the output path, using a small verifier rather than a frontier model.

Several widely recommended practices have weak or contrary evidence:

• Semantic chunking. Qu et al. (Findings of NAACL 2025) concluded the computational cost is not justified by consistent gains. The best-performing chunker in the EMNLP study was a plain sliding window.
• Adding random documents to the retrieved set. A 2024 SIGIR result suggested this improved accuracy, and it became a popular counter-intuitive talking point. A 2026 reproduction found the effect vanished on newer models under standard inference settings and was an artefact of the original configuration.
• Self-consistency, which as noted amplifies the positional problem it is meant to smooth over.
• Long context as a substitute for retrieval, per NoLiMa above.
• Context compression, which the best-practices study found could often be removed entirely with comparable results at lower latency.

The index is a correctness and a security boundary

Everything above assumes the corpus itself is right. Three findings suggest that assumption deserves more scrutiny than it usually gets.

Freshness is a correctness input, not a maintenance task. FreshQA (Vu et al., Findings of ACL 2024) showed models struggle on fast-changing knowledge unless augmentation supplies current material — and a retrieval corpus is only as current as its last successful ingest.

Rerankers do not know what "current" means. FRESCO, a 2026 preprint, paired recency-seeking queries with historical document revisions and found that when rerankers erred, they overwhelmingly ranked obsolete passages above valid ones — the best model tested still doing so in the large majority of error cases on queries that implied recency without stating it. The mechanism is intuitive once stated: an older, longer, more detailed document reads as more semantically relevant than a newer, terser replacement. Relevance is not recency, and nothing in a vector similarity score encodes the difference.

Write access to the index is write access to the answers. PoisonedRAG (USENIX Security 2025) achieved a 90 percent attack success rate by injecting five crafted documents per target question into a corpus of millions. The retrieval index is a trust boundary, and it is frequently treated as a cache.

Permissions belong in the same category. The 2025 revision of the OWASP Top 10 for LLM applications added vector and embedding weaknesses as a category in its own right, naming unauthorised access to embeddings, cross-context leakage between tenants, embedding inversion recovering source text, and poisoning. A retrieval corpus assembled without carrying over the access controls of its source systems will eventually answer a question it should have refused.

Where the storage layer fits

The conclusion most RAG programmes reach the hard way is that a substantial share of RAG hallucination is a data problem wearing a model costume. Ingestion loss, superseded versions, missing coverage, an unauditable index, permissions that did not survive the copy into the vector store — none of these are fixed by a better model, a cleverer prompt, or a longer context window.

What that asks of the layer underneath is fairly specific: hold the corpus at scale without fragmenting it, keep it current under continuous ingest while it is being queried, carry enough metadata to scope retrieval to the right subset, retain prior versions, and keep the source archive immutable so a citation still resolves to the document it cited.

Scality's platforms are positioned against that list. RING aggregates the stages of an AI pipeline into a single S3 data lake and integrates with established vector databases to serve RAG rather than attempting to be one; it supports extensible metadata tagging with both UI and API metadata search, which is what makes scoping a retrieval corpus to a subset practical at scale; and it supports S3 Object Lock in governance and compliance modes, the mechanism that makes a cited source archive tamper-evident across a retention window. ADI, announced in May 2026, is positioned across training, inference, agentic workflows and retrieval-augmented generation.

The questions worth pressing on any vendor here, Scality included, are not about throughput. They are about what happens to a retrieval corpus over time: how a superseded document is stopped from being retrieved as though it were current, whether the access controls of the source system survive the copy into the index, and whether you can reconstruct months later which version of which document an answer was actually grounded in. Those are storage questions, and they determine how often the model appears to be lying.