Prepare an incident runbook with containment, evidence, ownership, recovery, and restart checks.
Define what needs a response
An incident is a problem that needs coordinated action in your operating context. Define examples before an urgent report arrives.
Impact and scope guide the response. A rejected draft differs from incorrect information already sent to many recipients.
A reporting route must be easy to find and owned. NIST’s AI RMF includes feedback, monitoring, incident response, and recovery across deployed systems.
Contain the affected behavior
Containment limits further impact while investigation continues. Pause the affected sending step or route work to a tested manual process.
The pause owner needs real authority to act. Record a backup owner so response does not depend on one person being available.
A scoped response avoids assuming every component has failed. However, widen containment when evidence shows the problem crosses the original boundary.
Preserve useful evidence
An incident record captures timestamps, affected items, observed outputs, and relevant system versions. Separate confirmed facts from assumptions and unanswered questions.
Sensitive records need controlled access and appropriate retention. Do not copy every user message into a broadly shared incident document.
Execution evidence helps distinguish an attempted action from a completed one. Check the destination system before deciding what must be corrected or reversed.
Work through a fictional sending error
A fictional briefing service sends twelve summaries before an editor spots a retired deadline. The drafts cite an outdated policy version.
The immediate response pauses scheduled sends and preserves the affected output IDs. The team checks which messages were actually delivered before contacting recipients.
The recovery work corrects the source selection and prepares reviewed corrections. Reverting software alone cannot retract information people have already received.
| Stage | Named role | Evidence or action |
|---|---|---|
| Contain | Operations owner | Pause scheduled sends; confirm the queue is stopped |
| Scope | Editor and operations owner | Identify delivered messages and the source version |
| Correct | Content owner | Approve accurate replacement information |
| Repair | System maintainer | Fix version selection and preserve the change record |
| Retest | Evaluation owner | Test current, historical, and conflicting-source cases |
| Restart | Release owner | Review results and authorize a limited restart |
Treat restart as a new decision
A repair addresses the cause you identified. Retest the original failure and nearby cases before assuming the problem is resolved.
A restart record states what changed, who reviewed it, and which uncertainties remain. Begin within a scope that the team can actively monitor.
A learning review looks beyond individual mistakes. Ask which missing control allowed the error through and add that failure to the evaluation set.
Practice: rehearse the runbook
Your exercise imagines a wrong answer discovered after ten recipients received it. Write the first three actions and assign an owner to each.
A tabletop rehearsal walks through the scenario without sending real messages. Check whether the team can find the pause control, affected records, and correction owner.
A strong response separates containment, investigation, and recovery. Explain why rollback alone does not undo an external side effect.
Compare with the worked answer
Worked answer: First pause further sends and confirm that containment worked. Next preserve the relevant evidence and identify actual recipients. Then have the content owner approve a correction plan. Retest the repaired source selection before a named release owner authorizes restart.
Use the method on your own task
Editable worksheet: Download the Markdown template and open it in a text editor. Use fictional or approved information; no account or paid service is required.
Download the worksheet (.md) ↓What to carry forward
Prepare an incident runbook with containment, evidence, ownership, recovery, and restart checks.
Show your reasoning with the exercise above. Keep the evidence, the decision, and the remaining uncertainty together.