Create a small evaluation plan that exposes failures and supports a clear release decision.
Define the decision first
Evaluation should answer a decision: can this system support a particular task under stated conditions? Define the task, users, boundaries, and consequences of failure before choosing a score. A system that drafts internal notes needs different acceptance criteria from one that acts on customer accounts.
The NIST AI RMF 1.0 organizes risk work into Govern, Map, Measure, and Manage. These are connected functions, not a one-time linear checklist. Use them as prompts for ownership, context, evidence, and responses to risk.
Build a useful case set
Create representative examples of normal work, difficult cases, and situations where the correct behavior is to stop or ask for help. Include missing evidence, conflicting instructions, and unusual formatting. Record why each case belongs in the set.
Keep examples used for improvement separate from a held-back set used to assess changes. Repeatedly tuning to the same test cases can create a misleading impression of general quality. A small set is a starting point for learning, not proof of universal reliability.
Worked example: a source-based assistant
Fictional scenario. A team evaluates an assistant that answers questions from a handbook. Their rubric checks whether each answer uses the right passage, stays within its scope, and declines when the handbook does not answer the question.
A polished answer that invents a policy fails the evidence check. A clear “not covered in this handbook” can pass. Reviewers record the failure type and compare disagreements before relying on the scores. Formatting, factual support, and appropriate refusal remain separate dimensions.
Make the release rule explicit
Set acceptance criteria before comparing versions. Decide which failures block a release even if average performance improves. Compare the proposed version with the current baseline on the same cases, and inspect the changed answers.
Track latency and cost alongside quality when they affect the real task. After release, monitor a sample of actual work and maintain a route for users to report mistakes. Assign an owner who can pause or roll back the system if conditions change.
Keep a case-level evidence record
A case record includes the input, expected behavior, actual output, and separate check results. Save the configuration needed to interpret that result.
An inconclusive run may reflect a broken test harness or unavailable source. Do not quietly count infrastructure failures as model-quality passes.
A reviewer check tests whether the rubric describes the intended behavior. Resolve unclear expectations before interpreting disagreement as poor model performance.
Turn the result into a bounded decision
A release memo names the permitted use, observed failures, missing evidence, and decision owner. Separate planned checks from completed tests.
A limited pilot needs a defined audience, review arrangement, and stop condition. A smaller rollout still requires explicit responsibility.
A retest trigger may be a changed prompt, model, tool, source collection, or user population. Record which assumptions the change could invalidate.
Try it yourself
Write ten evaluation cases for the workflow in the previous guide: ordinary inputs, edge cases, and at least two cases requiring escalation. Define a short rubric and a release-blocking failure. Have another person independently score two outputs.
Success looks like: disagreements reveal a criterion you can clarify, and your release decision can be explained using recorded evidence rather than a single overall impression.
What to carry forward
Create a small evaluation plan that exposes failures and supports a clear release decision.
Show your reasoning with the exercise above. Keep the evidence, the decision, and the remaining uncertainty together.