The Core Idea: In classic RAG the system always retrieves once and then answers. In agentic RAG the agent decides whether to retrieve, rewrites the query, checks the results and tries again, so it handles harder questions.
- Beginner: Agent + Retrieval
RAG (Retrieval-Augmented Generation) gives a model relevant documents to read before answering, so it can use private or recent knowledge. An agent adds decision-making on top. Combining them gives an agent that can look things up when it needs to.
Simplest view: Retrieval becomes one more tool the agent can call, such as search_knowledge_base(query).
- Beginner: Retrieval as an Agent Tool
To the agent, retrieval is just a tool with a name, a description and parameters. It returns a few passages with their sources, and the agent reads them like any other tool result.
| Tool part | Example |
|---|---|
| Name | search_knowledge_base |
| Description | Search the company help documents. Use for policy and product questions. |
| Parameter | query (a short search phrase) |
| Returns | The top passages and their document names and scores |
Tip: Return scores and sources. Scores let the agent judge quality, and sources let it cite.
- Medium: Agent-Controlled Retrieval
Instead of retrieving blindly, the agent controls the process. It can:
In the code: Plain RAG always picks the top result even with a score of 0.0. The agent retries with a rewritten query and succeeds, or honestly says it could not find the answer.
- Medium: Knowledge-Based Agents
A knowledge-based agent answers from a defined body of knowledge, such as product manuals or company policies. Good ones follow four habits:
| Habit | Why it matters |
|---|---|
| Ground answers in retrieved text | Reduces invented facts |
| Cite the source document | Lets users verify |
| Say I could not find that when nothing relevant is found | Honesty beats a confident guess |
| Keep knowledge fresh | Re-index when documents change |
- Advanced: RAG vs. Agentic RAG
| Classic RAG | Agentic RAG | |
|---|---|---|
| Retrieval | Always once, with the user's question | When and as often as the agent decides |
| Query | The original question | Rewritten and refined |
| Quality check | None | Looks at scores and relevance |
| Multi-part questions | Weak | Searches several times |
| Cost and latency | Low, predictable | Higher, varies |
| Best for | Simple lookup questions | Complex, ambiguous or multi-step questions |
Choose wisely: Start with classic RAG. Move to agentic RAG only when you can show that simple retrieval fails on real questions, because it costs more and is harder to predict.
- Visual Explanation: Classic RAG and Agentic RAG Side by Side
The figure shows one question processed by classic RAG and by agentic RAG, then lists the five controls an agent has over retrieval.
Click to see why scores matterCompare ▼
Part 1 shows classic RAG using the wrong document while agentic RAG recovers. Part 2 lists the controls. Open the bar to see why scores matter.
- Python Implementation: Plain RAG vs. Agentic RAG
This script builds a tiny knowledge base and runs four questions through plain RAG (always use the top result) and agentic RAG (decide, rewrite, check the score, retry, or admit failure). Word overlap stands in for embeddings.
DOCS = { "refund": "Refunds are accepted within 30 days of purchase with a receipt.", "shipping": "Standard shipping takes 3 to 5 business days. Express takes 1 to 2 days.", "warranty": "All devices include a 12 month warranty covering manufacturing defects.",}STOP = {"the", "a", "an", "is", "of", "to", "my", "i", "can", "how", "what", "until", "does", "do", "for", "in"} def words(text): tokens = [w.strip(".,?!'").lower() for w in text.split()] tokens = [w[:-1] if len(w) > 3 and w.endswith("s") else w for w in tokens] return {w for w in tokens if w not in STOP} def retrieve(query): q = words(query) scored = [(len(q & words(text)) / max(len(q), 1), doc_id) for doc_id, text in DOCS.items()] return max(scored) def plain_rag(question): score, doc_id = retrieve(question) # always uses the top result, good or bad return doc_id, round(score, 2) SYNONYMS = {"arrive": "shipping", "send back": "refund", "guarantee": "warranty", "broken": "warranty"} def rewrite(question): extra = [target for phrase, target in SYNONYMS.items() if phrase in question.lower()] return question + " " + " ".join(extra) def needs_retrieval(question): return not question.lower().startswith(("hello", "hi", "thanks")) def agentic_rag(question, threshold=0.2, max_tries=2): if not needs_retrieval(question): return "no retrieval needed, answer directly" query = question for attempt in range(1, max_tries + 1): score, doc_id = retrieve(query) if score >= threshold: return "found '" + doc_id + "' (score " + str(round(score, 2)) + ") on attempt " + str(attempt) query = rewrite(query) # the agent reformulates and tries again return "could not find it, will say so honestly" for q in ["Hello!", "How long until my package arrives?", "Can I send back a gadget?", "What is the CEO's name?"]: print(q.ljust(38), "| plain RAG:", plain_rag(q), "| agentic RAG:", agentic_rag(q))Plain RAG returns the warranty document for every question, even with a score of 0.0. The agent skips retrieval for the greeting, finds the right document on the second attempt for shipping and refunds, and correctly admits it has no answer about the CEO.
Knowledge Check
Key Points
The Big Picture
Decide → search → check the score → rewrite and retry → answer with sources or say it is unknown
Retrieval is one of the most valuable tools an agent can have, and controlling it well is a skill of its own.
Up Next: Your agent can now act, remember and look things up. In Agent Reasoning and Decision Making you will study how it chooses what to do next.