agentic rag

Agentic RAG combines retrieval-augmented generation with the agent loop covered under agent, turning retrieval from a single fixed step into something the model can decide to repeat, refine, or skip based on what it actually finds. Standard RAG retrieves once — a query goes out, a fixed number of documents come back, and the model answers from whatever it got, even if those documents turn out to be irrelevant or incomplete; agentic RAG instead lets the model treat retrieval itself as a tool call it can reason about, inspect the results of, and decide whether to issue again with a refined query before it ever commits to an answer.

The failure mode standard RAG has no way to recover from is exactly the one agentic RAG is built to catch. A single fixed retrieval step has no mechanism for noticing that the documents it got back don't actually answer the question — the model is simply handed whatever came back and asked to make the best of it, which produces a confidently wrong answer whenever the first retrieval attempt misses. Letting the model inspect its own retrieval results, the way it inspects any other tool's output in the broader agent loop, means a bad first retrieval becomes a reason to retry with a better query rather than a dead end the model has to paper over.

Multi-hop questions are the clearest case where repeating retrieval actually earns its cost. A question like "what company did the founder of the firm that acquired Instagram previously work at" requires finding the acquirer, then the founder, then that founder's prior employer — three separate facts, each one only discoverable once the previous one is known, which a single retrieval pass against the original question has no way to assemble, since the intermediate entities never appear in the original query at all. Agentic RAG handles this by treating each hop as its own retrieval decision, informed by what the previous hop actually returned, rather than trying to retrieve everything needed in one shot.

The added retrieval attempts cost real latency and tokens, which is exactly why agentic RAG isn't the default choice for every retrieval task — simple, single-fact lookups rarely need a second attempt, and the extra reasoning overhead of deciding whether to retrieve again is pure cost on a question a single retrieval pass would have answered correctly anyway. The technique earns its place specifically on the harder end of retrieval tasks, where a fixed single-shot pipeline's biggest weakness is not having any way to notice, let alone fix, its own bad first attempt.