A useful AI agent kill switch is a coordinated containment path. It has to stop active work, refuse the next proposed action, close the identity path, isolate the workload, and preserve a record of what happened. A single control covers only one part of that job. This guide lays out the architecture, the failure modes, and the test evidence an operator should require. It follows the practical lesson from our walkthrough of the Hugging Face autonomous-agent intrusion: machine-speed action needs a control on the action path. General technical guidance only; adapt the design to your systems and threat model.
The term "kill switch" covers several controls
Identity platforms use the term for access containment. Okta's June 2026 support article says its current kill switch is a manual action and instructs an administrator to disable the agent record, linked application, and associated authorization server. Its regulated-environment release describes manual deactivation as the action that prevents new token requests and future authorizations.
Those controls close an identity path. The operating system may still have a worker in memory, a queue may still hold scheduled jobs, and a downstream service may already have accepted a request. A system-level stop therefore fans one incident command out to several independent enforcement points.
Treat the command as a versioned state change with an incident identifier, tenant, target scope, reason, actor, issued time, and deadline. Each enforcement point acknowledges the same command and reports its result. The incident stays uncontained while a required acknowledgement is missing.
| Enforcement point | Required action | Acknowledgement evidence |
|---|---|---|
| Execution orchestrator | Cancel active runs and remove queued work for the selected agent, Release, tenant, or environment | Run IDs, prior state, terminal state, cancellation time, unfinished activity |
| Policy enforcement point | Deny every new tool call and output release covered by the incident scope | Incident rule version, decision receipt, first denied action, deny count |
| Identity and secrets | Disable the agent identity, revoke tokens, rotate exposed credentials, and prevent refresh | Identity ID, token and key identifiers, revocation result, expiry of residual access |
| Workload and network | Terminate or quarantine workers and apply the incident egress policy | Workload IDs, isolation policy, connection inventory, quarantine time |
| Downstream systems | Reject the incident identity or idempotency keys and halt pending side effects | Rejected request IDs, pending jobs removed, committed side effects listed |
| Evidence store | Seal the command, acknowledgements, policy decisions, and gaps | Incident timeline, integrity result, exporter status, named evidence owner |
A network cutoff leaves several execution paths open
Network isolation is one actuator in the fan-out. It stops traffic crossing the boundary it controls. Local code can keep running, a database transaction may already be committed, an external job may continue after accepting the request, and a worker may hold a credential that remains valid on another route.
The safe boundary sits immediately before each side effect. The agent proposes a tool call. An enforcement point checks the incident state, identity, tool, arguments, policy, and current budget. The call proceeds only after that decision. A kill command updates the incident state that every later check reads.
This design also covers agents that ignore an in-process signal. The deciding component runs outside the agent process and owns access to the tool. Workload termination then shortens the time to containment while the external enforcement point prevents new effects.
- Local execution: terminate or quarantine the process, container, virtual machine, or sandbox
- Queued execution: cancel scheduled messages and invalidate leases held by workers
- Credentialed execution: revoke or rotate every credential reachable from the affected scope
- Accepted downstream work: cancel pending jobs through the downstream API and list the jobs that already committed
- Human-approved work: close pending Decision Requests and prevent a late approval from resuming the run
The stop sequence
The sequence below minimizes the gap between the operator action and the last possible side effect. The first durable write establishes the incident epoch. Every policy enforcement point reads that epoch before executing a governed call.
| Order | Command | Completion condition |
|---|---|---|
| 1 | Create the incident command and set the affected scope to deny | The durable incident epoch is readable by every enforcement point |
| 2 | Signal active runs and cancel queued work | Each run is terminal or explicitly listed as still executing |
| 3 | Disable identities, revoke tokens, and rotate exposed secrets | New authorization fails and residual token lifetime is recorded |
| 4 | Quarantine workloads and restrict egress | Affected compute has no unapproved route to a tool or data store |
| 5 | Cancel pending downstream jobs and start compensating actions | Every accepted request has a completed, pending, failed, or compensating state |
| 6 | Seal the containment record and expose missing acknowledgements | The incident owner can see the final side effect, remaining exposure, and evidence gaps |
How the KLA stop path behaves
KLA Control Plane uses separate KLA Runtime and KLA Policy Engine controls. A responder with incident management permission can freeze a running execution from its incident. The freeze sends a kill signal to the run’s workflow, which records the run as CANCELLED. The workflow checks the signal before it starts another node. A run waiting on a Decision Request observes the signal, cancels the pending request, and exits without resuming the proposed action. When the run owns a pending agent action, the worker cancels that action and KLA seals a cancellation receipt for it.
The general execution cancel route closes only orphaned records whose workflow is confirmed absent, and it cannot stop a healthy run. Drills and runbooks should use the incident freeze. Freezing an agent from an incident pauses its stages in Processes: each agent step checks the freeze before it runs, then hands the work to a person or stops the run.
Agents that act through KLA agent action ingress work under an agent mandate. Revoking the mandate blocks open proposals bound to it with the reason code mandate_revoked, cancels their pending Decision Requests, and refuses the next action the agent proposes. An action that was already admitted may complete. The agent pause does not apply on this path, so revoke the mandate to stop an ingress agent.
When enabled, the KLA Policy Engine governance gateway evaluates proposed tool calls and enforces allow, warn, require_approval, or block. If policy evaluation fails, the gateway defaults to block in production and require_approval in other environments, and never to allow. An approval binds to a hash of the tool arguments, and changed arguments are blocked. The gateway cleanup operation for held tool calls records cancelled ledger entries and seals cancellation receipts. KLA calls it when a pending agent action is cancelled, so operators must verify any other held entries separately during containment.
An activity that already crossed its enforcement gate may finish before the workflow observes the signal. Downstream idempotency, workload termination, and compensating actions cover that interval. The incident record must state the last side effect that committed after the operator issued the command.
This section describes the current KLA source code.
The Accountable Autonomy guide explains where human decisions belong in ordinary operation. During containment, the incident scope has priority and a late approval cannot reopen a cancelled run.
Design every degradation mode
The kill path is most valuable while part of the system is failing. Each actuator needs a local safe state, a deadline, and a visible acknowledgement status.
| Failure | Required behavior | Evidence to retain |
|---|---|---|
| Policy service unreachable | Apply the local fail-closed decision and keep the incident epoch in the enforcement cache | Reason code, cache version, denied action, recovery time |
| Workflow signal delayed | Deny new side effects at the enforcement point and quarantine the workload | Signal attempts, workflow state, quarantine acknowledgement |
| Identity provider unavailable | Apply downstream deny rules, isolate the workload, and expire short-lived credentials | Provider error, residual credential lifetime, systems under local deny |
| Evidence exporter unavailable | Keep the source records durable and flag the bundle as pending | Source record locations, integrity state, retry owner |
| A side effect already committed | Open the system-specific compensation or recovery procedure | Original request, committed result, compensation owner and result |
| One actuator never acknowledges | Keep the incident uncontained and escalate to the named system owner | Missing actuator, elapsed time, escalation history |
Test the switch as a control system
A dashboard response confirms receipt of the operator request. Acceptance requires evidence from every enforcement point and an observed upper bound on the final side effect.
NIST AI RMF Manage 2.4 and Manage 4 call for mechanisms to disengage or deactivate AI systems whose outcomes diverge from intended use, plus documented incident response, recovery, and communication. The NIST Generative AI Profile adds named ownership, rehearsal, retrospective improvement, and alignment with reporting law for third-party AI incident plans.
- Active run: issue the command while a multi-step run is between tool calls and verify the next call is denied
- Approval wait: issue it while a Decision Request is open and verify a later approval cannot resume the run
- Policy outage: disconnect the policy service and verify no proposed action resolves to allow
- Stale credential: attempt access with every token class issued before the command and record residual lifetime
- Queued work: seed scheduled jobs and confirm each one reaches a terminal or compensating state
- Uncooperative process: ignore the in-process signal and verify external enforcement and workload isolation still contain it
- Evidence failure: stop the exporter and confirm the source record remains durable with a visible retry owner
- Recovery: restore one Release under a fresh identity and verify the incident deny remains on every affected older version
Run a kill-switch drill
A drill turns the stop path into measured times. Run it in a test environment with a synthetic agent that holds the same kinds of rights as the production agent: a running execution, a pending Decision Request, a held tool call, and a live credential. Name a drill lead, an observer who keeps the clock, and the owner of each downstream system.
A kill switch stops future actions. A payment that settled, an email that was delivered, or a record that was overwritten before the command stays done. The drill finds those effects and hands each one to the correction procedure of the system that owns it.
We learned why in our own dogfood testing. A responder took Freeze agent through the full containment flow, with a reason code, an explanation, and a review of blast radius and reversibility, and committed it. The request failed with an error the screen did not show, and the incident still read not contained. The incident referenced the run, and the freeze needed the agent. We changed the action to resolve the agent from the run, or to return an error that names the action to use instead. Step 2 of the drill checks each run's terminal state for this reason: a responder must know whether containment happened.
- Restore only when the cause is known or contained, every in-flight action has a state, each completed effect has a correction owner, and a person other than the requester approved the new authority.
- Keep the old authority revoked. The restored agent runs under a new mandate or Release, and a test shows the revoked one still refuses.
- Raise monitoring for the restored agent and set the threshold that ends the heightened watch.
| Step | Action | Pass condition | Record |
|---|---|---|---|
| 1. Declare | Open the incident, set the scope (agent, Release, tenant, or environment), and start the clock at T0 | Incident and scope exist before any actuator runs | Incident ID, scope, declarer, T0 |
| 2. Freeze runs | Freeze each active execution in scope and freeze the agent for Process stages | Every run in scope is CANCELLED or listed as still executing | Run IDs, prior state, terminal state, time |
| 3. Revoke authority | Revoke the agent mandate, disable the identity, and revoke tokens at the identity provider | New authorization fails and residual token lifetime is known | Revocation actor, reason, time, token IDs and expiry |
| 4. Attempt the next action | Have the synthetic agent propose its next action on every path it used: ingress, gateway, and a direct call with an old token | Every attempt is refused and none reaches the downstream system | Attempt time, path, refusal code such as mandate_revoked, downstream log |
| 5. Classify in-flight work | List every action that was running or held at T0 | Each action is completed before T0, completed after T0, cancelled, or pending, with no unknown state | Action ID, state, commit time, owner |
| 6. Try a late approval | Approve the Decision Request that was pending at T0 | The approval cannot resume the cancelled run | Decision attempt and refusal |
| 7. Restore | Issue a new mandate or Release under fresh credentials once the restoration conditions hold | Only the new authority works; the revoked mandate still refuses | Restoration approver, conditions checked, first allowed action |
| 8. Close | Seal the drill record and list each gap with an owner | The incident owner signs the record | Evidence reference, gaps, remediation owners, next drill date |
Sample drill record (synthetic)
The record below is synthetic. It shows the level of detail a drill should keep, including the findings a drill exists to surface.
| Time (UTC) | Event | Result |
|---|---|---|
| 09:00:00 | Drill lead declares incident INC-DRILL-07 for one synthetic payments agent | Scope recorded |
| 09:00:40 | Responder freezes run R-118 (running) and run R-121 (waiting on a Decision Request) | Both runs CANCELLED by 09:00:44; the R-121 Decision Request is cancelled |
| 09:01:10 | Compliance officer revokes the agent mandate with the reason "drill" | Two open proposals blocked with mandate_revoked |
| 09:01:30 | Identity owner disables the agent identity and revokes its tokens | An access token issued at 08:55 stays valid at the target API until 09:10 |
| 09:02:05 | Synthetic agent proposes payments.create_transfer through ingress | Refused with mandate_revoked; no request in the payment system log |
| 09:02:20 | Synthetic agent calls the target API directly with the 08:55 token | Accepted by the target API. Finding: close the residual token window in the target system |
| 09:03:00 | Observer classifies in-flight work | One transfer committed at 09:00:41 inside R-118 before the run read the signal. Finding: treasury owns the recall request |
| 09:04:30 | Checker tries to approve the R-121 request that was pending at T0 | Refused because the request is no longer pending |
| 09:20:00 | Drill lead seals the record with two findings and an owner for each | Next drill scheduled after both fixes |
Connect the switch to the incident playbook
The kill switch establishes containment. The responder still has to scope the incident, preserve volatile records, restore a known-good Release, compensate for committed effects, and decide which notifications apply.
Use the AI agent incident response playbook for that sequence. Feed its thresholds from the post-market monitoring plan so the same signals that open the incident can invoke a rehearsed stop path.
Frequently Asked Questions
What is an AI agent kill switch?
An AI agent kill switch is a coordinated incident command that cancels active runs, denies new actions at the enforcement point, revokes credentials, isolates affected workloads, cancels pending downstream work, and records which controls completed. The scope may target one run, agent, Release, tenant, or environment.
Can token revocation stop a rogue AI agent?
Token revocation closes the authorization paths governed by those tokens. Runtime cancellation, workload isolation, queue cancellation, downstream deny rules, and compensating actions cover work that is already executing or already accepted.
Where should the kill-switch check run?
Run the decisive check outside the agent process and immediately before every side effect. The check should read a durable incident state and deny calls within the affected scope, including calls proposed by a process that ignores its own cancellation signal.
How do we test an AI agent kill switch?
Test active runs, approval waits, queued work, stale credentials, a policy outage, an uncooperative process, an evidence-export failure, and recovery under a known-good Release. Measure the time and number of side effects between the command and the final committed action.
What evidence should the switch produce?
Keep the incident command, actor, scope, issue time, acknowledgements from each actuator, denied actions, token and key changes, workload isolation result, final committed side effect, pending compensation, and integrity status of the evidence record.
Key Takeaways
A credible AI agent kill switch is a small control system with several independent actuators. Put a durable deny state on the action path, cancel the workflow, close identity access, isolate compute, reconcile downstream work, and keep the incident open until every required control acknowledges. Rehearsal turns that architecture into a measured containment time and a record an incident owner can defend.