Explain the difference between generating a useful answer and establishing that it is true.
The mental model
Machine learning uses data to train models that make predictions or generate content. A language model produces text from patterns learned during training and the context supplied at use time. Treat its response as a candidate answer that still needs a task-appropriate check.
Separate three layers: the model, the application around it, and the information available for the current task. A chat application may add search, files, or tools. Those capabilities belong to the system; do not assume that every model can browse or inspect your documents.
Give the task a boundary
A useful prompt names the goal, audience, available evidence, desired format, and limits. Start with a task small enough to inspect. “Summarize these meeting notes into decisions and open questions” is easier to evaluate than “manage this project.”
Context is the material available in a particular interaction. Supplying a document does not guarantee that every relevant detail will appear in the answer. Ask for references to specific passages and compare them with the original.
Worked example: a meeting summary
Fictional scenario. Your notes say: “Maya will investigate two suppliers. The budget is still undecided.” A generated summary says: “Maya will purchase from the preferred supplier next week.” The sentence sounds plausible, but adds an approval, a preference, and a deadline.
A better instruction is: “Use only these notes. Separate agreed actions from unresolved questions. Do not infer deadlines or approvals.” A better result records an investigation as the action and the budget as unresolved. Verify both against the notes before sharing.
Common mistakes
- Confusing fluent language with evidence.
- Assuming a model knows current facts without checking how the application retrieves information.
- Changing many prompt instructions at once, then being unable to explain what improved.
Improve one instruction at a time. Keep the original input and compare the results against the same criteria.
Follow the answer through the system
The input contains your request and any material the application supplies. First identify what was actually provided rather than what you expected it to know.
The model response is one stage in the process. A search result, calculator result, or saved note may come from a separate component.
Your verification should target the relevant component. Check retrieved facts against sources and calculations against a calculation, rather than judging the whole system by tone.
Use a simple acceptance check
A supported summary preserves the difference between a decision and a proposal. It should retain important uncertainty from the original notes.
A completeness check looks for missing details as well as invented ones. Removing every difficult statement can produce a safe-sounding but useless summary.
A final comparison places the source and output side by side. Record one preserved fact, one correction, and one remaining uncertainty.
Try it yourself
Write six lines of fictional meeting notes containing two decisions, one proposed idea, and one unresolved question. Ask for a summary. Highlight every statement that cannot be traced to the notes. Rewrite the prompt to reduce those additions.
Success looks like: you can explain which statements are supported, which are missing, and which were invented. Save your observations in the Beginner module notes.
What to carry forward
Explain the difference between generating a useful answer and establishing that it is true.
Show your reasoning with the exercise above. Keep the evidence, the decision, and the remaining uncertainty together.