RAG Evolution: Naive, Advanced and Modular

See how RAG grew from a simple retrieve-and-generate pipeline into advanced and modular designs, where each approach fails, and how to choose between them.

25 minIntermediateCode Examples
The Core Idea: RAG systems did not appear fully formed. They grew in three steps. Naive RAG is the simple pipeline: index, retrieve, generate. Advanced RAG adds improvements before, during and after retrieval. Modular RAG turns the pipeline into interchangeable parts that can be rearranged, repeated or skipped. Knowing this path tells you what to build first and what to add only when a measured problem demands it.

1. Beginner: Naive RAG

Naive RAG is the pipeline from the first lessons, with no extras: split documents into chunks, embed and store them, embed the question, fetch the top-k chunks, put them in a prompt, and let the model answer.
Index chunks→Embed the question→Fetch top-k→Build prompt→Generate
Strengths: simple to build, easy to understand, and surprisingly useful for clean documents and direct questions. It is the right starting point and the baseline that every improvement must beat.

2. Intermediate: Where Naive RAG Fails

Each weakness of naive RAG maps to a technique you have met in this course.
SymptomLikely causeFix, and where you learned it
Nothing found for a clear questionQuery wording differs from documentsQuery transformation
Wrong or outdated answerOld versions and unrelated chunks are retrievedMetadata filtering and versioning
Right chunk retrieved but ranked lowRough first-stage scoringReranking
Exact codes and names missedEmbedding-only searchHybrid retrieval
Question needs several factsOnly one search is doneMulti-step retrieval
Confident answer with no supportNo check on retrieval or outputGrounding and verification

3. Intermediate: Advanced RAG

Advanced RAG keeps the same skeleton but upgrades each phase.
Before retrieval
Better chunking, rich metadata, query rewriting, expansion and decomposition.
During retrieval
Hybrid dense and sparse search, metadata filters, tuned top-k.
After retrieval
Reranking, deduplication, compression, careful ordering and grounded prompts.
The pipeline is still mostly a fixed straight line. It is stronger than naive RAG, but every question takes the same path, and every improvement is hard-wired into it.

4. Intermediate: Modular RAG

Modular RAG treats each capability as a module with a clear input and output: query rewriter, retriever, reranker, memory, router, verifier, and so on. An orchestrator decides which modules run, in what order, and how many times.
PatternWhat happensExample
LinearModules run in a fixed orderNormalize, retrieve, rerank, generate
ConditionalA router picks the pathSimple question skips retrieval; hard one uses multi-step
LoopingSteps repeat until a check passesRetry retrieval if the answer is unsupported
ParallelSeveral branches run and are mergedSearch documents and a database together
Why it matters: modules can be swapped, tested alone and removed. This turns system design into experiments: take one module out and measure what you lose.

5. Advanced: Choosing an Architecture

More machinery means more cost, delay and failure modes. A sound approach is to start simple and add parts only in response to measured failures.
1. Build naive RAG and an evaluation set of real questions.
2. Find the most common failure and add the one module that addresses it.
3. Re-measure. Keep the module only if quality improves enough to justify its cost.
4. Repeat, and move toward a modular design once different questions clearly need different paths.
SituationReasonable choice
Small, clean knowledge base, direct questionsNaive RAG, perhaps with hybrid search
Mixed question types and measurable failuresAdvanced RAG with targeted upgrades
Many sources, tools, or questions needing different pathsModular RAG with routing and checks
A warning: modularity is not a goal in itself. A system with ten modules and no evaluation is harder to trust than a simple one that you can measure.

6. Visual Explanation: Three Generations of RAG

Panel A places the three generations side by side. Panel B shows measured results from the code below. Panel C is a symptom explorer.
Panel A · Naive, Advanced, ModularEach generation adds control
Generation 1
Naive RAG
Index, retrieve, generate in one straight line.
Generation 2
Advanced RAG
Upgrades before, during and after retrieval.
Generation 3
Modular RAG
Swappable modules with routing and loops.
Panel B · Hit Rate on Four QuestionsFrom the code in section 7
Naive RAG
1 of 4 questions answered by the right document
Without normalize
2 of 4
Without active filter
3 of 4
Advanced and modular, both modules
4 of 4
Panel C · Symptom ExplorerClick a symptom to open it
Click to see why you should not build everything at once▼
Each module adds latency, cost and new ways to fail. Without a baseline and a test set you cannot tell which additions help, so grow the system one measured step at a time.
How to read it: Panel A is the family tree, Panel B shows that each module earns its place on test questions, and Panel C maps problems to additions.

7. Python Implementation: Naive vs Advanced vs Modular with Ablations

The program defines five tiny documents, one of which is an outdated vacation policy, and four test questions with known answers. Naive RAG searches once with the raw question. The advanced and modular pipelines are built from two modules: a normalizer that expands abbreviations such as PTO and WFH, and a filter that keeps only active documents. Because the modules are plain functions in a list, the program can remove one at a time and measure the result.
rag_evolution.pyPython 3.10+ · Standard library only
rewrites = dict([("pto", "paid time off"), ("wfh", "work from home"), ("2fa", "two factor authentication")])stop_words = set(["how", "many", "do", "to", "when", "what", "is", "a", "the"]) documents = [    ("d1", "active", "vacation policy 2025 employees receive 20 days of paid time off"),    ("d2", "superseded", "vacation policy 2023 employees get 15 days of paid time off"),    ("d3", "active", "work from home policy remote work two days per week"),    ("d4", "active", "security policy enable two factor authentication"),    ("d5", "active", "expense policy submit receipts within 30 days"),] tests = [    ("how many PTO days do employees get", "d1"),    ("WFH rules", "d3"),    ("2FA requirements", "d4"),    ("when to submit receipts", "d5"),] def words(text):    return "".join(ch if ch.isalnum() or ch == " " else " " for ch in text.lower()).split() def top1(query, docs):    q = set(w for w in words(query) if w not in stop_words)    scored = [(len(q & set(words(text))), doc_id) for doc_id, status, text in docs]    scored.sort(key=lambda s: (-s[0], s[1]))    return scored[0][1] if scored and scored[0][0] > 0 else None # Naive RAG: one search with the raw questiondef naive(query, docs):    return top1(query, docs) # Modules: each takes (query, docs) and returns (query, docs)def normalize(query, docs):    return " ".join(rewrites.get(w, w) for w in words(query)), docs def only_active(query, docs):    return query, [d for d in docs if d[1] == "active"] def run(modules, query, docs):    for module in modules:        query, docs = module(query, docs)    return top1(query, docs) def score(label, fn):    hits = sum(1 for question, gold in tests if fn(question) == gold)    print(label.ljust(34), "hit rate:", hits, "/", len(tests)) score("Naive RAG", lambda q: naive(q, documents))score("Advanced: normalize + filter", lambda q: run([normalize, only_active], q, documents))print("Modular ablations (remove one module):")score("  without normalize", lambda q: run([only_active], q, documents))score("  without active filter", lambda q: run([normalize], q, documents))score("  both modules", lambda q: run([normalize, only_active], q, documents))
Expected output
Naive RAG                          hit rate: 1 / 4Advanced: normalize + filter       hit rate: 4 / 4Modular ablations (remove one module):  without normalize                hit rate: 2 / 4  without active filter            hit rate: 3 / 4  both modules                     hit rate: 4 / 4
Naive RAG answers one question out of four. Each module fixes a different failure: normalization rescues the questions that use abbreviations, and the active filter prevents the outdated vacation policy from winning. Removing either module lowers the score, which is exactly the experiment that modular design makes easy. Advanced and modular pipelines score the same here, because the benefit of modularity is flexibility and testing, not magic.

Knowledge Check

Q1. What is the main difference between advanced and modular RAG?
Q2. A team sees outdated policies in answers. Which addition addresses this most directly?
Q3. What is a sound way to evolve a RAG system?

Key Points

✓
Naive RAG indexes, retrieves and generates in a straight line and is the baseline for every improvement.
✓
Naive RAG fails on poor queries, noisy or outdated chunks, weak ranking, exact terms, multi-fact questions and unsupported answers.
✓
Advanced RAG upgrades the phases before, during and after retrieval.
✓
Modular RAG builds the system from swappable modules arranged linearly, conditionally, in loops or in parallel.
✓
Modules make ablation tests possible: remove one and measure the loss.
✓
Add complexity only in response to measured failures, since each module costs latency and money.
✓
Match the architecture to the situation: simple for clean small data, modular for varied questions and sources.

The Big Picture

The three generations are less a ladder to climb than a toolbox to grow into. The right design is the simplest one that meets your quality goals on real questions, and the habit of measuring each change is what lets you stop at the right place.
Naive baseline → Measure failures → Add targeted modules → Orchestrate → Re-measure
Up Next: Corrective, Self and Adaptive RAG. You will study RAG systems that check their own retrieval and answers: corrective RAG, self-reflective RAG, adaptive routing and retry loops.