Glossary
Large language model (LLM)
A large language model (LLM) is an AI system trained on very large amounts of text to predict the next token, a word or part of a word, given the text before it. What it learned is held in billions of numeric weights. Training fixes them, so its knowledge ends at a cutoff date and excludes private data.
Generation is prediction, not lookup
Text is split into tokens. At each position the model scores every possible next token given everything before it, picks one, and repeats. Fluent paragraphs come out of that loop, not out of a store of facts that is consulted. The loop runs the same way whether or not the model saw the relevant facts in training, so when they are missing it still produces a plausible answer. The model also reads only a limited window of text per request, counted in tokens, which is why retrieval supplies selected passages rather than whole document collections.
What the weights leave out
Two kinds of knowledge are absent by construction: anything that happened after the training cutoff, and anything held in private systems. The familiar result is an assistant that discusses the wider world with confidence and gets the organization's own products, prices and policies wrong. It never saw them. Retraining only moves the cutoff forward; private data stays out unless it is deliberately supplied, through one of two routes.
Two routes for company data
| Route | What changes | Suited to |
|---|---|---|
| Retrieval at question time | The text supplied with each question | Facts that change, answers that need sources, per-user permissions |
| Fine-tuning | The model weights, through extra training | House style, output format, a fixed task or behavior |
The common mistake is fine-tuning to teach facts. The facts change, the model has to be trained again, and each round leaves another set of weights to evaluate and keep. The choice turns on what is changing, as set out under RAG vs fine-tuning: changing facts point to retrieval, stable behavior points to training.
What does an LLM put on storage?
Training and fine-tuning write checkpoints in large bursts. Model versions accumulate, since each release or fine-tune is kept for rollback and audit. Copies spread across platforms as teams experiment. Retrieval adds a document corpus, its embeddings and an index, and a change of embedding model means reading the whole corpus again. These pressures fall on the layers described under AI data infrastructure.
Model and retrieval system are separate parts
The model generates language. The retrieval system, built on a vector database or search index, supplies it with information. Most enterprise assistants combine the two, and when answers are wrong the fault lies as often in retrieval as in the model, as covered under RAG hallucination.
Frequently asked questions
What is a token?
A unit of text the model reads and writes, often a word or part of one. Prompt length, cost and response time are all counted in tokens.
Does an LLM store the documents it was trained on?
Not as documents. It holds statistical patterns in its weights. It can sometimes reproduce passages that appeared often in training, but it is not a searchable archive.
Can an LLM run entirely inside an organization?
Yes, with open-weight or licensed models deployed on its own infrastructure. That keeps prompts and documents under its control and also places the compute and storage load on it.