Glossary
RAG storage
RAG storage is the infrastructure that holds every data layer of a retrieval-augmented generation system, from the original documents to the logs of questions and answers. Only two of those layers, the sources and the logs, cannot be regenerated, and those two decide how the rest is protected and sized.
Five layers, two of them irreplaceable
| Layer | What it holds | Can it be rebuilt? | Usual home |
|---|---|---|---|
| Source documents | Original files in every format | No | Versioned object storage |
| Extracted text | Text parsed from PDFs, slides and scans | Yes, from the sources | Beside the sources |
| Chunks and metadata | Passages tagged with source, version and access rules | Yes | Document store or database |
| Search indexes | Embeddings plus a keyword index | Yes | Memory or flash |
| Query and answer logs | What was asked, retrieved and answered | No | Retained, locked store |
The middle three layers behave as cache: losing them costs compute and time, not knowledge. The sources are the only copy of what the organization knows, and the logs are the only record of what the system told people. A log that can settle a dispute records the question, the identity of the person asking, the passages retrieved with their document versions, and the answer given; a log of questions and answers alone cannot show why the system said what it said. Those two get the protections applied to business records, meaning versioning, retention periods and a lock against early deletion such as S3 Object Lock.
Re-indexing sets the read requirement
Embeddings from different models cannot be compared, so replacing the embedding model invalidates the whole index. A new chunking rule does the same. Either change means every source document is read, chunked and embedded again. Query traffic touches a small share of the corpus; a re-index touches all of it. Sizing the document store for queries alone therefore understates its real load, and bulk storage throughput decides how long an embedding upgrade takes to finish.
Small indexes over a large, growing corpus
The indexes are compact and answer lookups in milliseconds, so they sit on fast media. The document store is large, varied and grows without a ceiling, and it serves follow-up reads when the full text of a cited source is needed. Keeping the two apart lets each sit on media suited to its job: an index can be lost and rebuilt, but it has to answer fast, while the document store can be slower per request as long as it holds everything durably. Storage tiering moves older, rarely retrieved material to cheaper capacity. When agentic RAG turns one question into many small reads, response time per request starts to matter more than bandwidth.
Why do audits and erasure requests reach every layer?
Personal data can sit in a source file, its extracted text, a chunk, an embedding and a logged question, so a request to erase it has to be traced through all five layers. Audits run in the other direction: reproducing a disputed answer needs the log entry and the source documents as they stood on that day, which only versioned sources can supply. RAG storage is a narrower view than AI storage, which also covers training data and model checkpoints; the pattern these layers serve is described under retrieval-augmented generation.