Human Oversight for AI Agents: When Is Approval Required?

The four-outcome approval rule¶

Evaluate the proposed side effect before it reaches the tool or downstream system. Apply every relevant rule, then keep the strongest result. A block cannot be weakened by a matching allow rule, and a required approval cannot disappear because the amount happens to be below one financial threshold.

The outcome describes execution behavior. allow releases the action inside current authority. warn releases it and creates a defined follow-up. require_approval holds the exact action until an authorised human decides. block prevents the action from reaching the governed side effect.

Map the seven approval inputs to policy¶

Thresholds belong to the organisation that owns the action. Start with one action inventory and define testable bands for all seven inputs. Financial amount alone is insufficient: a zero-value entitlement change or disclosure of restricted data can carry more risk than a large reversible transfer between controlled accounts.

Keep the inputs as separate rules. A combined score can hide one decisive fact. A prohibited destination must stay blocked even when every other input looks routine.

Human in the loop, on the loop, and in command¶

These labels describe operating arrangements. They are useful design terms and are not outcomes defined by the EU AI Act. Pick the arrangement that gives the accountable person enough time, context, and authority for the action risk.

Use maker-checker separation for consequential actions¶

The maker creates or sponsors the request. For an agent action, the maker record should identify the agent, its accountable owner, the requesting principal, and the exact proposed side effect. The checker is a distinct, qualified person with authority for that action class. The checker reviews the evidence and chooses approve, reject, request changes, or escalate.

Enforce separation at decision time. A group name in a workflow definition does not prove that the person who acted was eligible or independent. Resolve the current identity, role, delegation, conflicts, and request origin when the decision is made.

• Bind the request. Hash or otherwise bind the action, parameters, destination, policy inputs, and reviewer evidence so approval cannot be replayed for a changed request.
• Resolve eligibility. Check the reviewer’s identity, active role, authority limit, training status, and any conflict or requester relationship.
• Make independent judgment possible. Show source facts, uncertainty, limitations, policy reasons, alternatives, and downstream consequences without preselecting approval.
• Record one decision. Capture identity, role snapshot, decision, reason, rationale reference, time, and the evidence digest the checker saw.
• Revalidate before release. Reject stale approval when the action, evidence, policy, identity, authority, destination, or relevant business state changed.
• Attach the outcome. Preserve the actual execution receipt and downstream effect under the same execution and decision identifiers.

Give the reviewer the context needed to decide¶

A useful Decision Request answers the decision in one short narrative, with structured detail available for verification. The reviewer should understand what will happen, why policy routed the action, which facts remain uncertain, what authority they hold, and when the request becomes stale.

Set service levels, expiry, delegation, and reassignment¶

A service level is an operating target. Expiry is an authorization boundary. Set both from consequence, reversibility, volatility of the evidence, and the time available to prevent harm. No universal duration fits every action.

An emergency stop must remain immediately available to an authorised operator and must not wait behind an approval queue. For ordinary requests, the example bands below are starting points for a policy workshop. Replace them with system-specific values and test them under real staffing conditions.

• Delegation: grant a named person a narrow action class, value limit, environment, effective period, and escalation route. Preserve who delegated the authority.
• Reassignment: require an eligible replacement, record the previous assignee and reason, and keep the original evidence and expiry visible.
• Expiry: invalidate the approval capability. Never turn queue timeout into implicit approval.
• Staleness: expire early when material inputs, policy, identity, destination, amount, or requested parameters change.
• Unavailable control: hold or block according to the action’s fail-closed policy and page the responsible operator.

Design overrides, appeals, and emergency stops as separate paths¶

An override changes a decision or output under explicit authority. An appeal asks a different authorised role to review a decision. An emergency stop interrupts active or queued operation and brings the system to a defined safe state. Combining the three into one admin button obscures authority, timing, and evidence.

Recognise approval workflow failure modes¶

An approval screen can exist while the control fails. Test the full path from request creation through the downstream effect, including outages, retries, workload spikes, and recovery.

• Rubber-stamping: reviewers approve too quickly or repeat identical reasons. Measure latency, agreement, overrides, and reviewer load; use independent samples.
• Self-approval: the requester, agent sponsor, or conflicted operator can approve. Resolve identities and conflicts at decision time.
• Stale approval: evidence, amount, destination, policy, or parameters change after review. Bind and revalidate the request before execution.
• Orphaned wait: no eligible reviewer owns the item or the queue loses it. Monitor assignment, ageing, escalation, and expiry as control health.
• Replay or duplicate execution: one approval releases several calls or a retry repeats the side effect. Use single-use decisions and idempotency keys.
• Partial downstream effect: a multi-step action fails after one external commitment. Record each effect and run the compensation or incident path.
• Evidence written too late: the side effect commits before the policy or approval record is durable. Fail closed when the required record cannot be written.
• False stop: the interface says stopped while workers, queues, credentials, or retries continue. Test propagation and reconcile every in-flight operation.

Worked example: a synthetic credit disposition¶

The public AI Agent Audit Log Schema includes a complete synthetic record for a credit-review agent proposing a EUR 24,000 disposition update. The example shows how approval connects to the actual downstream effect. The figures and identities are synthetic.

Use a minimum evidence schema¶

Use stable identifiers and machine-readable fields so a reviewer or auditor can join the decision to execution without relying on timestamps or screenshots. KLA publishes a vendor-neutral JSON Schema, examples, and verifier for this record.

How KLA implements the approval control path¶

The KLA Control Plane governs instrumented agent actions. The KLA Policy Engine evaluates a proposed tool call against published rules and returns allow, warn, require_approval, or block. A require_approval outcome holds the proposed call and creates a Decision Request for Decision Desk. A block prevents the governed call from reaching the tool.

For control-plane Decision Requests, Decision Desk checks decision permission, the required reviewer role, pending state, requester and maker identity, and due time. It prevents recorded requesters and makers from deciding their own request and rejects approve or reject actions after the due time. An eligible checker can escalate an overdue request. It records the decision actor, role snapshot, outcome, reason, and time. These separation and expiry controls rely on the request carrying current maker identities and a due time. Ensure every producing path supplies them and test the full control path.

Lineage Explorer and Audit Trail expose the policy, human-decision, and execution records. Evidence Room can package selected records into a Sealed Evidence Bundle with signatures, artifact hashes, and a Merkle root that supports offline integrity checks. The organisation still owns action classification, reviewer competence, staffing, legal analysis, complete instrumentation, source completeness, intervention across every worker and downstream system, and the safe-state design.

Primary sources and freshness¶

Source review completed 28 July 2026. EU AI Act Article 14 requires effective human oversight for high-risk AI systems and names monitoring, automation-bias awareness, interpretation, disregard, override, reversal, intervention, and safe interruption capabilities. Article 26(2) requires deployers of high-risk systems to assign oversight to people with the necessary competence, training, authority, and support. Recital 73 explains the role of informed intervention and built-in operating constraints.

Regulation (EU) 2026/1744 changed the application dates for the Chapter III high-risk rules to 2 December 2027 for Article 6(2) and Annex III systems and 2 August 2028 for Article 6(1) and Annex I systems. It did not replace the Article 14 control text.

The NIST AI RMF Core calls for differentiated human-AI roles, documented oversight processes, independent review, and mechanisms for appeal, override, decommissioning, incident response, and recovery. NIST Appendix C notes that the need for human oversight depends on context. The OECD AI Principle on human-centred values calls for human agency and oversight safeguards appropriate to context and state of the art.

The four-outcome table, seven-input framework, service-level examples, and evidence schema in this guide are implementation patterns. They carry no universal legal threshold. Reconfirm applicable law, regulatory guidance, sector rules, system facts, and current source text before relying on them.