Glossary
RAG vs fine-tuning
RAG vs fine-tuning compares two ways of giving a language model knowledge or behavior it lacks. Retrieval-augmented generation supplies relevant documents with each question and leaves the model unchanged. Fine-tuning continues training on curated examples, producing new weights that need another training run for every update.
The two approaches side by side
| Question | RAG | Fine-tuning |
|---|---|---|
| What is changed | The text supplied with each question | The model weights |
| Adding or correcting knowledge | Update the document and re-index it | Prepare examples, retrain, evaluate, redeploy |
| Citing a source | Yes, the retrieved passage is known | No, the knowledge is blended into the weights |
| Per-user access control | Applied when passages are retrieved | Not possible inside one shared model |
| Main data needed | Source documents and an index | Curated examples and training capacity |
Retrieval-augmented generation changes the input. Fine-tuning changes the model itself, and the result is deployed as a new model version.
Facts go to retrieval, behavior goes to training
Fine-tuning shapes how a model responds: tone, output format, domain vocabulary, or a classification task with a fixed set of labels. These are stable properties that never need a citation. Retrieval handles knowledge that changes, needs a source or must be hidden from some users. A support assistant shows the split: a tuned model writes replies in the house ticket format, and retrieval supplies the warranty terms in force today. The two combine routinely; they are not rival choices.
What happens when a fact changes?
With retrieval, a changed policy is a changed file. It is chunked, embedded and indexed, and the next question sees it. With fine-tuning, the same change means building new examples, running training, evaluating the result and deploying another model. There is also no clean way to remove one learned fact, and facts absorbed in training are recalled inconsistently and cannot be traced to a source. Knowledge that changes monthly is the usual reason teams separate the two. Withdrawal is the sharper case: a document pulled for legal reasons disappears from retrieval at the next index update, while a fact trained into weights stays until the next model replaces it.
Each approach leaves different data behind
Retrieval depends on source documents and an index, the layers described under RAG storage, and its load is mostly reads. Fine-tuning produces example sets, training checkpoints and a new set of weights per run, each kept for rollback and audit, and its training runs lean on GPU storage. Both start from an already trained large language model; training one from scratch is a separate and far larger undertaking.
How a poor fit shows up
A tuned model asked about a rule that changed after training quotes the old rule with full confidence. A retrieval system asked to answer in a strict format drifts from it, because no document teaches style. The first is fixed by moving facts into retrieval, the second by tuning or by clearer instructions. Retrieval-side failures, where the documents exist and the answer still departs from them, are covered under RAG hallucination.