Explain a RAG pipeline and diagnose a missing, outdated, or unsupported answer.
Understand the basic idea
Retrieval-augmented generation brings selected external information into the model’s context when answering a question. It does not, by itself, retrain the model.
The retrieval step searches a collection for relevant material. Then the generation step uses that material to produce a response.
A useful distinction separates finding evidence from using it correctly. Therefore, evaluate those two jobs separately when an answer goes wrong.
Follow the evidence
Document preparation often divides files into smaller passages called chunks. Preserve titles, dates, and section labels so passages keep their meaning.
Search methods can use matching words, semantic similarity, or both. Exact identifiers and broadly phrased questions may need different retrieval behavior.
Source references help readers inspect the evidence used in an answer. However, a displayed citation does not prove that the cited passage supports the claim.
Worked example: the wrong handbook
Fictional scenario: a staff assistant answers a question using last year’s equipment handbook. Its response is faithful to that document but wrong for today’s process.
The diagnosis starts with the retrieved passages and their dates. In this example, the failure concerns source selection rather than sentence generation.
A better design labels document versions and excludes retired guidance from ordinary answers. If current sources conflict, the assistant should expose the conflict for review.
Know what RAG does not solve
Missing evidence remains missing even when search returns something similar. A safe answer may explain that the available collection does not cover the question.
Access permissions must apply before restricted passages reach the model. A final instruction to hide confidential information is not a substitute for access controls.
System evaluation needs both answerable and unanswerable questions. Also test whether a relevant passage is retrieved but misread in the final answer.
Try it yourself
Your exercise uses three short fictional documents with dates and titles. Write five questions, including one the documents cannot answer.
A manual simulation selects the relevant passages before drafting each answer. Record what evidence was missing and whether the draft added unsupported details.
Success looks like a clear diagnosis of retrieval failure versus answer failure. Mark the source passage that supports each conclusion.
What to carry forward
Explain a RAG pipeline and diagnose a missing, outdated, or unsupported answer.
Show your reasoning with the exercise above. Keep the evidence, the decision, and the remaining uncertainty together.