Retrieval and Its Limits: Building RAG Properly, Then Finding Its Ceiling

Retrieval-augmented generation is the most requested skill in AI engineering and the most casually taught. This course builds it properly, measures it honestly, and then shows you the wall it hits.

You will write a naive pipeline in sixty lines with no framework, tune it into a real one with hybrid retrieval and reranking, put a recall@k number on your own corpus, and then run a seeded-defect evaluation that demonstrates the ceiling: retrieval cannot detect a missing prerequisite, because similarity is a property of the neighbourhood and completeness is a property of the artifact.

Seven lessons, about three hours, a graded quiz on each. Free and open to access.

What you will learn

• Place LangChain, RAG, MCP, CLIs and skills on one diagram and say what each actually does
• Build a working retrieval pipeline without a framework, then tune it with hybrid search and reranking
• Measure recall@k against a labelled set instead of trusting a vibe check
• Compare LangChain against plain Python on line count, token cost and time to debug
• Demonstrate the failure class that retrieval cannot catch, and diagnose it in writing

Course curriculum

7 lessons, 3 hours, a graded quiz on each.

1. Three Things People Confuse: A Protocol, a Technique and a Library Artifact: Reference diagram
2. The Naive Build: Sixty Lines, No Framework Artifact: Working pipeline
3. Retrieval Done Properly: Chunking, Hybrid Search and Reranking Artifact: recall@k number on your corpus
4. The Same Pipeline Twice: LangChain Versus Plain Python Artifact: Side-by-side comparison table
5. The Transfer Problem: Why Your Pipeline Is Tuned to a Model Artifact: Before and after eval scores
6. The Ceiling: What Retrieval Structurally Cannot Detect Artifact: Failure demonstration
7. Assessment: Write the Diagnosis Artifact: Written diagnosis

Gate

Pipeline running, recall@k measured on your own corpus, written diagnosis submitted.