An AI agent incident compresses the normal response cycle. The system can propose another tool call, refresh a credential, or start another run while the team is still opening a bridge. This playbook gives the incident commander an ordered path: contain the active capability, preserve the record, restore a known-good Release, reconcile side effects, and make the reporting decision from verified facts. It maps the evidence to EU AI Act Article 73 without assuming that every agent or incident falls within that provision. General operational information only; qualified counsel should confirm legal scope, deadlines, and notifications.
Open one incident and start the clocks
The NIST Generative AI Profile recommends defined ownership, regular rehearsal, retrospective improvement, and alignment with breach-reporting, privacy, and other laws for third-party generative AI incident plans. NIST SP 800-61 Revision 3 integrates incident response across preparation, detection, response, and recovery.
Declare one incident record before responders make parallel changes. Record the first detection time, awareness time, affected system and Release, observed behavior, known side effects, people affected, environments, agent and human identities, and the source of each fact. Keep assertions, hypotheses, and decisions as separate fields.
Assign four named roles. The incident commander owns sequence and containment. The system owner operates the agent and downstream services. The evidence lead preserves records and the action timeline. The legal owner decides notification scope and coordinates with counsel, privacy, security, and sector teams.
- Severity owner: sets the current impact level and the next review time
- Containment deadline: sets the latest acceptable time for every kill-switch actuator to acknowledge
- Evidence owner: records source, collection time, integrity state, and custody for every artifact
- Reporting owner: records each possible regime, statutory trigger, authority, clock, and decision
Preserve volatile evidence before broad changes
Capture volatile state as containment begins: active run identifiers, queued jobs, open Decision Requests, process and workload identity, current Release, model and prompt version, policy version, credential identifiers, network connections, tool-call ledger, and downstream request IDs. Seal the collection time and collector identity.
For a covered serious incident, Article 73(6) of the EU AI Act requires the provider to investigate, assess risk, and take corrective action. It also requires the provider to inform the competent authorities before changing the AI system in a way that may affect a later evaluation of the incident's causes. Counsel should decide when that condition applies. The playbook should make the evidence-preservation checkpoint explicit so containment and regulatory preservation can proceed together.
Copy the relevant logs into an incident-controlled store with retention protection. Preserve the raw records and create derived timelines separately. Hashes and signatures can show whether the collected files changed after capture; they cannot prove that an omitted source never existed, so the collection manifest must list expected sources and missing sources.
Contain the capability
Invoke the AI agent kill-switch architecture at the narrowest scope that contains the observed behavior. Expand to the agent, Release, tenant, or environment as facts change.
Containment ends when every required actuator acknowledges or the incident commander records a manual substitute. A green status from the workflow service cannot establish containment for tokens, queued downstream jobs, or a worker running outside that service.
| Target | Action | Fact required for containment |
|---|---|---|
| Active and gated runs | Send cancellation, close pending Decision Requests, and deny later resume attempts | Terminal state and last committed side effect for each run |
| Policy path | Install an incident-scoped block at every tool and output gate | Policy version, first denied action, and fail-closed behavior |
| Agent identity | Disable the identity, revoke tokens, prevent refresh, and rotate exposed secrets | Residual access window and result for every credential class |
| Workload | Terminate or quarantine compute and restrict egress | Workload inventory and isolation acknowledgement |
| Queues and downstream jobs | Cancel pending work and list requests that already committed | Terminal or compensating state for every accepted request |
| Data access | Freeze affected write paths and preserve snapshots where authorized | Stores covered, snapshot time, and writes observed after declaration |
Roll back the system and reconcile the effects
Choose the recovery point from evidence. Record the last known-good Release, policy pack, model and prompt version, tool manifest, connector configuration, and secret generation. Verify their provenance before promotion.
A Release rollback restores the previous deployed bytes and configuration. It cannot recall an email, undo a completed transfer, restore a deleted external record, or retract data already disclosed. Open a system-specific compensation item for every committed effect and assign an owner and deadline.
Bring recovery up under fresh credentials and an incident-specific allowlist. Run a controlled canary with the incident deny still active for affected older versions. Release broader traffic only after the canary produces the expected policy decisions, side effects, and evidence.
- Release: restore the verified prior artifact and retain the failed artifact for analysis
- Policy: restore the verified prior policy while keeping the incident-scoped deny at higher precedence
- Identity: issue fresh credentials with a new generation and the minimum required scope
- State: restore from an authorized snapshot only after the evidence lead records its provenance and impact
- External effects: compensate, notify, or manually correct each committed action through the owning system
- Monitoring: raise sampling and alerting for the recovered Release and define the exit threshold
Build the incident evidence record
The record should let an independent reviewer reconstruct the event without reading a chat transcript or asking the responders to remember it. Preserve the source event sequence and attach decisions as separate, signed records.
In KLA Control Plane, Lineage Explorer reconstructs the agent run, Audit Trail keeps governance and operator actions, and Evidence Room packages selected records as a Sealed Evidence Bundle. These paths are implemented in the KLA development environment, which is the only environment KLA currently runs. The bundle still needs a collection manifest that names expected sources, missing sources, and the scope chosen by the evidence owner.
| Evidence group | Required fields | Question answered |
|---|---|---|
| Detection | Signal, threshold, source, first observed time, first triage, awareness decision | How and when did the organization know |
| Execution | Run, agent, Release, model, prompt, tool calls, arguments, outputs, side effects | What capability acted and what changed |
| Governance | Policy versions, decisions, Decision Requests, approvers, roles, argument hashes | Which controls evaluated each consequential action |
| Containment | Command, scope, actor, acknowledgements, final side effect, remaining exposure | When did the capability stop |
| Recovery | Known-good provenance, Rollback, canary, fresh identity, compensation items | How was safe operation restored |
| Reporting | Applicable regimes, trigger analysis, causal assessment, counsel decision, submissions | Why, when, and where was the incident reported |
Map the facts to EU AI Act Article 73
The official EU AI Act text limits Article 73 to providers of high-risk AI systems placed on the Union market. Article 3(49) defines a serious incident as an incident or malfunction that directly or indirectly leads to death or serious harm to health, serious and irreversible disruption of critical infrastructure, infringement of Union-law obligations protecting fundamental rights, or serious harm to property or the environment.
The Digital Omnibus on AI, Regulation (EU) 2026/1744, entered into force on 27 July 2026. It changed the application dates for the high-risk requirements in Chapter III to 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems. Article 73's reporting windows remain unchanged. The interaction between system classification, provider or deployer role, sector law, transition dates, and Article 73 requires a case-specific legal decision.
Record that decision with the facts available at the time. A security event outside Article 73 scope may require action under another regime.
| Provision | Statutory rule | Playbook record |
|---|---|---|
| Article 73(1) | A provider of a high-risk AI system placed on the Union market reports a serious incident to the market surveillance authorities of the Member States where it occurred | System classification, provider identity, market placement, Member States, authorities |
| Article 73(2) | Report immediately after establishing a causal link or reasonable likelihood of one, with a ceiling of 15 days after awareness | Awareness time, causal assessment versions, report time, reason for elapsed time |
| Article 73(3) | A widespread infringement or serious and irreversible disruption of critical infrastructure has a ceiling of two days after awareness | Impact category, geographic scope, awareness time, two-day deadline |
| Article 73(4) | A death is reported immediately after causation is established or suspected, with a ceiling of 10 days after awareness | Known harm, suspicion time, causal evidence, ten-day deadline |
| Article 73(5) | An incomplete initial report may be followed by a complete report when needed for timeliness | Initial submission, known gaps, update owner, complete-report target |
| Article 73(6) | The provider investigates, assesses risk, takes corrective action, cooperates, and informs authorities before a change that may affect later cause evaluation | Investigation plan, preserved state, corrective actions, authority contact before material change |
| Article 26(5) | A deployer that identifies a serious incident immediately informs the provider first, then the importer or distributor and relevant market surveillance authorities | Role analysis, contact attempts, notification order, unreachable-provider path |
Use a 24-hour operating sequence
Set the internal response pace well inside the statutory reporting ceilings so legal review receives verified facts while evidence is still available.
| Elapsed time | Primary objective | Exit evidence |
|---|---|---|
| 0 to 15 minutes | Declare incident, assign roles, set deny scope, invoke kill switch | Incident ID, awareness time, command, actuator status |
| 15 to 60 minutes | Preserve volatile state, revoke identity, isolate workload, scope downstream work | Collection manifest, credential result, final known side effect |
| 1 to 4 hours | Select known-good state, Rollback, open compensation items, run controlled canary | Recovery provenance, canary result, compensation register |
| 4 to 8 hours | Reconstruct the event and classify harm, role, geography, and possible regimes | Versioned timeline, impact assessment, legal issue list |
| 8 to 24 hours | Approve notifications, issue the first status update, define monitoring and investigation plan | Reporting decision, submissions, stakeholder update, next review time |
Rehearse the gaps that appear under pressure
NIST AI RMF Manage 4 calls for documented and monitored response, recovery, and communication plans. The post-market monitoring plan should supply the thresholds, ownership, and evidence feeds that open this playbook.
Run exercises for an active run, an approval wait, a stolen credential, a committed external side effect, a missing log source, an unreachable provider, and an unavailable evidence exporter. Record containment time, the final post-command side effect, missing acknowledgements, evidence completeness, recovery time, and reporting-decision time.
Use the Accountable Autonomy guide to confirm that ordinary oversight and emergency authority have named owners. Review the Hugging Face incident walkthrough for a public example of machine-speed action, credential harvesting, lateral movement, and reconstruction from more than 17,000 recorded events.
Frequently Asked Questions
What is the first action in an AI agent incident?
Declare one incident, assign the incident commander, system owner, evidence lead, and legal owner, record the awareness time, and set the affected scope to deny. Invoke the kill-switch actuators while preserving volatile execution, identity, policy, and downstream state.
How do we contain a rogue AI agent?
Cancel active and gated runs, deny new tool calls at an external enforcement point, disable the agent identity, revoke and rotate credentials, isolate the workload, cancel queued and downstream work, and keep the incident open until every required actuator acknowledges.
What does rollback mean for an AI agent incident?
Restore the verified prior Release, policy, model, prompt, tool manifest, and connector configuration under fresh credentials. Track every committed external side effect through a system-specific compensation, correction, or notification procedure.
When does EU AI Act Article 73 require a report?
Article 73 addresses providers of high-risk AI systems placed on the Union market when a serious incident under Article 3(49) occurs. The reporting clock and authority depend on causal assessment, incident type, Member State, role, sector overlap, and current application dates. Qualified counsel should confirm the case-specific obligation.
What are the Article 73 reporting deadlines?
Article 73 sets immediate reporting after a causal link or reasonable likelihood is established, with a 15-day ceiling after awareness. A widespread infringement or serious and irreversible critical-infrastructure disruption has a two-day ceiling. A death has a ten-day ceiling and an immediate trigger when causation is established or suspected. An incomplete initial report is permitted when needed for timeliness.
What evidence should an AI agent incident record contain?
Keep detection and awareness times, run and Release identifiers, model and prompt versions, tool calls, arguments, policy decisions, approvals, side effects, the containment command and acknowledgements, credential changes, preserved logs, Rollback provenance, compensation results, legal analysis, and reports.
Key Takeaways
A strong incident playbook establishes control before the team tries to explain the event. Stop the capability, preserve the state, restore a verified Release, reconcile every committed effect, and make the reporting decision from a versioned evidence record. Connect the playbook to post-market monitoring and rehearse it until containment time, evidence completeness, and recovery state are measured facts.