Retrieval and Its Limits: Building RAG Properly, Then Finding Its Ceiling

Retrieval-augmented generation is the most requested skill in AI engineering and the most casually taught. This course builds it properly, measures it honestly, and then shows you the wall it hits.

You will write a naive pipeline in sixty lines with no framework, tune it into a real one with hybrid retrieval and reranking, put a recall@k number on your own corpus, and then run a seeded-defect evaluation that demonstrates the ceiling: retrieval cannot detect a missing prerequisite, because similarity is a property of the neighbourhood and completeness is a property of the artifact.

Seven lessons, about three hours, a graded quiz on each. Free and open to access.

What you will learn

• Place LangChain, RAG, MCP, CLIs and skills on one diagram and say what each actually does
• Build a working retrieval pipeline without a framework, then tune it with hybrid search and reranking
• Measure recall@k against a labelled set instead of trusting a vibe check
• Compare LangChain against plain Python on line count, token cost and time to debug
• Demonstrate the failure class that retrieval cannot catch, and diagnose it in writing

Course curriculum

7 lessons, 3 hours, a graded quiz on each.

1. Three Things People Confuse: A Protocol, a Technique and a Library Artifact: Reference diagram
2. The Naive Build: Sixty Lines, No Framework Artifact: Working pipeline
3. Retrieval Done Properly: Chunking, Hybrid Search and Reranking Artifact: recall@k number on your corpus
4. The Same Pipeline Twice: LangChain Versus Plain Python Artifact: Side-by-side comparison table
5. The Transfer Problem: Why Your Pipeline Is Tuned to a Model Artifact: Before and after eval scores
6. The Ceiling: What Retrieval Structurally Cannot Detect Artifact: Failure demonstration
7. Assessment: Write the Diagnosis Artifact: Written diagnosis

Gate

Pipeline running, recall@k measured on your own corpus, written diagnosis submitted.

Frequently asked questions

Do I need a framework to do RAG?

No, and the course proves it. Lesson 2 builds a working pipeline in about sixty lines of plain Python. Lesson 4 then puts LangChain side by side with it and measures the difference, including where LangGraph genuinely earns its place.

What is recall@k and why does the course insist on it?

It is the proportion of relevant documents that appear in the top k retrieved results, measured against a labelled set. Without it you are tuning by anecdote.

What is the ceiling the title refers to?

Retrieval finds things similar to your query. It cannot tell you that something required is absent, because absence has no neighbourhood. Lesson 6 demonstrates this with seeded defects.