Glossary

Agentic RAG

Agentic RAG is retrieval-augmented generation in which the model plans its own retrieval, deciding what to search for, in which source and how many times. Standard RAG runs one search per question. An agentic system keeps searching until it judges the evidence sufficient, then answers with citations.

How the loop runs

In standard retrieval-augmented generation the path is fixed: one search of one index, and the top passages go to the model. A question such as which open incidents involve a contract that expires this quarter spans a ticket system and a contract store, and a single search returns a thin, partial answer. An agentic system works through it in a loop:

  1. Break the question into sub-questions: which contracts expire this quarter, and which open incidents name those customers.
  2. Choose a source for each sub-question.
  3. Run the searches, often in parallel.
  4. Read the results and judge whether the evidence answers the question.
  5. Search again with a refined query if it does not, or stop if it does.
  6. Write the answer, citing every passage used.

The plan, search, check and repeat cycle is the whole difference from the standard pattern.

Reads multiply per question

One user question becomes several searches, and each search becomes several reads of indexes and documents. The count varies with the question and the agent, so no fixed multiple applies. What holds in every case is that reads are small, scattered and numerous, which makes per-request storage latency more important than raw bandwidth.

Three risks that grow with each step

  • Access leaks. An agent running on broad service credentials can surface documents the person asking may not see. Permissions are applied with that person's identity on every sub-search, not once per question.
  • Runaway loops. An agent that never judges the evidence sufficient keeps searching. A step limit and a time limit turn an open loop into a bounded one.
  • Audit volume. Every sub-question, source and passage is a decision someone may need to reconstruct, so the log grows with the number of searches and needs a defined retention period.

When is one search enough?

The deciding factor is whether answers span systems or depend on an earlier result. Lookups such as finding the paragraph that states a policy stay cheaper and more predictable with one retrieval and a good hybrid search. Questions about relationships across many documents have another route, GraphRAG, which links facts at indexing time instead of searching repeatedly at question time.

Frequently asked questions

Is agentic RAG the same as an AI agent?

An AI agent is any model that chooses and uses tools. Agentic RAG is the case where the tool is retrieval, so the agent decides how to search the organization's data.

Does agentic RAG reduce wrong answers?

It can on questions that need several sources, because the agent can notice missing evidence and search again. Each extra step is also another place for retrieval or reasoning to go wrong.