The Core Idea: A language model only knows what it learned during training. Retrieval-Augmented Generation (RAG) gives it an open book: before answering, the system searches your documents, finds the most relevant passages, and hands them to the model together with the question. The model then writes an answer based on that evidence instead of relying on memory alone.
1. Beginner: The Open-Book Exam
Key point: RAG does not make the model smarter and does not change its parameters. It changes what the model gets to read at the moment it answers.
2. Beginner: What RAG Is in Three Steps
Analogy: a librarian (the retriever) fetches the right books from the shelves, and a writer (the LLM) reads those books and composes the answer. Neither could do the whole job as well alone.
3. Beginner: RAG vs a Traditional LLM
| Question | Traditional LLM | RAG system |
|---|---|---|
| Where does knowledge come from? | Training data only | Training data plus retrieved documents |
| Can it use private documents? | No, it never saw them | Yes, if they are in the knowledge base |
| How do you update its knowledge? | Retrain or fine-tune the model | Add or edit documents |
| Can it show its sources? | Not reliably | Yes, it can point to the passages used |
| What extra parts are needed? | None | Document store, search, prompt building |
Trade-off: RAG buys freshness, privacy and traceability, and pays for it with a more complex system. The next lesson explores exactly why that trade is usually worth it.
4. Intermediate: RAG Terminology
| Term | Plain meaning |
|---|---|
| Knowledge base | The collection of documents the system can search. |
| Chunk | A small piece of a document, such as a few paragraphs, stored and searched as one unit. |
| Embedding | A list of numbers that represents the meaning of a text, so similar meanings get similar numbers. |
| Vector store | A database that keeps embeddings and can quickly find the closest ones to a query. |
| Query | The user question that the system must answer. |
| Retriever | The component that finds the most relevant chunks for a query. |
| Top-k | The number k of best-matching chunks the retriever returns. |
| Context | The retrieved text placed inside the prompt for the model to read. |
| Generator | The LLM that writes the final answer from the prompt. |
| Grounding | Making sure the answer is supported by the retrieved context. |
| Hallucination | A confident-sounding answer that is false or unsupported. |
5. Intermediate: Basic Use Cases and Their Limits
Advanced note: RAG is only as good as its retrieval. There are two main ways it fails. If the retriever misses the right passage, the model has no evidence and may guess. If the retriever finds the right passage but the model ignores or misreads it, the answer is still wrong. Most of the later lessons in this course are about preventing these two failures.
6. Visual Explanation: Plain LLM vs RAG, Step by Step
Click to see what happens before any question arrivesExplain ▼
7. Python Implementation: A Tiny RAG Pipeline from Scratch
import mathfrom collections import Counter # 1. The knowledge base (four tiny documents)documents = [ "The company refund policy allows returns within 30 days of purchase.", "Standard shipping takes 5 to 7 business days inside the country.", "Premium members get free shipping on every order.", "Our support team is available Monday to Friday from 9 am to 5 pm.",] def tokenize(text): cleaned = "".join(ch for ch in text.lower() if ch.isalnum() or ch == " ") return cleaned.split() def vectorize(text): return Counter(tokenize(text)) def cosine(a, b): common = set(a) & set(b) dot = sum(a[w] * b[w] for w in common) norm_a = math.sqrt(sum(v * v for v in a.values())) norm_b = math.sqrt(sum(v * v for v in b.values())) if norm_a == 0 or norm_b == 0: return 0.0 return dot / (norm_a * norm_b) # 2. RETRIEVE: score every document and keep the top kdef retrieve(question, docs, k=2): q = vectorize(question) scored = [(cosine(q, vectorize(d)), d) for d in docs] scored.sort(reverse=True) return scored[:k] # 3. AUGMENT: put the context and the question into one promptdef build_prompt(question, retrieved): lines = ["Answer using ONLY the context below.", "", "Context:"] for score, text in retrieved: lines.append("- " + text) lines.append("") lines.append("Question: " + question) return lines question = "What is the refund policy for returns?"retrieved = retrieve(question, documents, k=2) print("Retrieved passages:")for score, text in retrieved: print(round(score, 3), "|", text)print("")print("Prompt sent to the LLM:")for line in build_prompt(question, retrieved): print(line)Retrieved passages:0.456 | The company refund policy allows returns within 30 days of purchase.0.114 | Standard shipping takes 5 to 7 business days inside the country. Prompt sent to the LLM:Answer using ONLY the context below. Context:- The company refund policy allows returns within 30 days of purchase.- Standard shipping takes 5 to 7 business days inside the country. Question: What is the refund policy for returns?Knowledge Check
Key Points
The Big Picture
Up Next: Why RAG? LLM Knowledge Limitations. You will look closely at the problems RAG solves, namely knowledge cutoffs, hallucinations, private data and changing information, and compare RAG with fine-tuning and long context windows.