Glossary

Retrieval-augmented generation (RAG)

Retrieval-augmented generation (RAG) finds the passages in an organization's own documents that are most relevant to a question and hands them to a language model along with that question. The model answers from that evidence, with sources, and is never retrained.

How does retrieval-augmented generation work?

The idea was set out in a 2020 paper from Facebook AI Research, now Meta, and the pattern has since become the default way to connect a large language model to private data. It runs in two phases. In preparation, documents are split into passages (see document chunking), each passage is turned into an embedding, a list of numbers that captures its meaning, and the embeddings are stored in a vector index. At question time, the question becomes an embedding too, the index returns the closest passages, and the model writes an answer from them and cites where each part came from.

The model's own knowledge stops at its training date and contains nothing about internal policies, tickets or contracts. Retrieval supplies that missing knowledge at the moment of the question, so the answer reflects the documents as they are today rather than as they were when training ended.

Which parts of a RAG system can be rebuilt?

Only one layer cannot. The source documents are the original, and everything downstream is derived from them: extracted text, passages, embeddings and the index. A change of embedding model or chunking rule makes the old index obsolete, and the whole corpus is read and processed again. Teams that treat the index as the system of record find out the cost when the first re-index arrives.

That makes read throughput on the document store as relevant as query speed on the index. It also explains why most RAG failures are data problems rather than model problems: stale indexes, text extracted badly from PDFs and tables, and exact codes that similarity search misses. The storage side of this is covered under RAG storage.

Why does RAG need permissions checked at retrieval?

A language model cannot be told to hide a passage it has already received. If retrieval returns a document the asking user is not entitled to read, the answer can quote it. Access rules therefore have to be applied when passages are fetched, using the identity of the person asking, and not only on the original file share. In shared environments this is the same isolation question that multi-tenant storage raises for data, now applied to an index.

Where do the other RAG terms fit?

Each related term changes one step of this flow. Semantic search is the matching step and hybrid search adds keyword matching beside it. GraphRAG and agentic RAG change how evidence is gathered for harder questions. RAG versus fine-tuning concerns where knowledge lives, and RAG hallucination describes what happens when any step fails. A curated collection of documents prepared for this use is an AI knowledge base.

Frequently asked questions

Does RAG retrain or change the model?

No. The model's weights stay as they are. Only the text placed in front of it changes from one question to the next.

Does RAG remove the need for a vector database?

Not usually. Some index is needed to find passages quickly, and for meaning-based matching that is a vector index. Small collections can use simpler stores, and many systems add keyword search beside the vector index.

Can RAG cite its sources?

Yes, because the passages handed to the model are known. Each can carry a document name and version, so an answer can point to the exact source it drew on.