Produce a vendor decision record that distinguishes demonstrated results, claims, and unresolved questions.
Start with the job, not the demo
The decision boundary is the specific workflow you are considering. Our fictional support team needs draft replies grounded in an approved policy library; automatic sending is outside scope.
A demonstration can suggest useful capabilities, but selected examples do not show performance on your ordinary work or difficult exceptions.
A requirement should describe observable behavior: cite the current policy, preserve its exceptions, and defer when the source is missing. Name the person who will check it.
Separate gates from preferences
A gate is a condition that must be met before a proposed use proceeds. For this fictional pilot, sending messages without review is prohibited and account isolation must be demonstrated.
A preference can be traded against another benefit, such as a shorter drafting time or a simpler interface. Write those trade-offs separately from gates.
An unknown is not a pass. A vendor statement may identify what to investigate, but it does not replace the required evidence. These example gates are local design choices, not universal procurement rules.
Compare the same task
| Criterion | Supplier A | Supplier B |
|---|---|---|
| Source-supported drafts | 18 / 20 | 17 / 20 |
| Required exceptions preserved | 3 / 5 exception cases | 5 / 5 exception cases |
| Reviewer-only sending | Demonstrated in pilot configuration | Demonstrated in pilot configuration |
| Cross-account access check | Passed tested cases | Not yet tested |
| Export and deletion | Documentation received; rehearsal pending | Rehearsal pending |
Exception cases are a subset of the twenty tasks, not an additional denominator. These small, invented samples illustrate reasoning; they do not establish future performance or compare real products.
A leads on the overall draft result but misses two required exceptions. B preserves the exceptions, yet its isolation check remains unknown. Neither is ready for the intended release on this evidence alone.
Keep claims and observations distinct
An evidence register records the requirement, evidence type, version, date, reviewer, result, and unresolved question. Label a sales statement as a claim and a completed rehearsal as an observation.
Configuration matters because a test with one permission setting or source collection does not validate every deployment. Record the actual settings and any dependencies on another service.
A follow-up request should ask for something reviewable: a controlled account-isolation test, an export sample, or documentation covering the exact service tier under consideration.
Count the operating work
Total effort includes review, source maintenance, integration, incident handling, and eventual replacement. A lower usage price alone does not establish a lower workflow cost.
Our fictional team records time spent correcting exceptions separately from ordinary drafting. If that work is hidden, a faster first draft can look like a saving while total handling time rises.
A pilot comparison uses the same definition of a completed, acceptable task in the existing process and the proposed process. Keep unfinished or rejected outputs visible.
Plan the exit before the commitment
An exit rehearsal asks whether the team can export necessary records in a usable form and continue the workflow without the service. Test a small approved sample rather than assuming an export button is enough.
A deletion question covers data you control and external copies whose retention needs confirmation. Do not confuse removing a local record with verified deletion throughout a supplier’s systems.
A decision record names the accountable owner, allowed use, open issues, evidence expiry, and a trigger to reconsider. A model, contract, permission, or integration change can invalidate earlier assumptions.
Practice: write the bounded decision
Using the fictional table, decide whether either supplier can enter the intended release. Then propose one narrower next step, the evidence required, its owner, and a stopping condition. Download the worksheet to keep your reasoning.
Compare with the worked answer
Worked answer: Do not approve the intended release yet. Ask A to repair exception handling and repeat those cases plus held-out cases. Ask B to complete controlled isolation testing. Both still need an export rehearsal. A restricted pilot with fictional data and no sending capability could gather evidence if the responsible owner accepts that boundary. Do not call either supplier the winner merely by averaging the table.
Use the method on your own task
Editable worksheet: Download the Markdown template and open it in a text editor. Use fictional or approved information; no account or paid service is required.
Download the worksheet (.md) ↓What to carry forward
Produce a vendor decision record that distinguishes demonstrated results, claims, and unresolved questions.
Show your reasoning with the exercise above. Keep the evidence, the decision, and the remaining uncertainty together.