Intermediate level · Module 4: Retrieval-Augmented Generation · Lesson 3 of 3
Evaluating Retrieval and Grounding
Measure whether the right passages are found and whether answers stay faithful to them.
In this lesson you will
- Measure retrieval hit rate
- Check answers for grounding and correct citations
Key concepts
retrieval hit rategroundingcitation accuracyunanswerable questions
The lesson
Evaluate retrieval separately: for each test question, is the passage containing the answer among the retrieved chunks? This hit rate shows whether failures come from search or from generation.
Check grounding: every claim in the answer should be supported by a cited passage. Unsupported claims are hallucinations even if they happen to be true.
Test unanswerable questions. A good assistant says it does not know and points to a human contact instead of inventing an answer.
Step by step
- Label the correct passage for each test question.
- Measure the retrieval hit rate.
- Check answers for grounding and citations.
- Test five unanswerable questions.
Practice activity
Evaluate and improve the FAQ assistant's retrieval and grounding.
Expected result: Measured hit rate and grounding results, with at least one improvement made.
Check your understanding
-
Why evaluate retrieval separately from answers?
Show answer
To find out whether failures come from search or generation.
-
What should the assistant do with an unanswerable question?
Show answer
Say it does not know and point to a human contact.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.