Enterprise-wide audit of responsibility, authority, execution, oversight, and evidence across AI agent systems. A production audit follows the complete operating system around an agent: the business purpose, model and orchestration, human and non-human identities, delegated authority, data and tools, policy decisions, approvals, state changes, business outcomes, monitoring, incidents, releases, rollback, retention, and independent verification. The method below gives internal audit, compliance, model risk, security, and operations teams one fieldwork structure while preserving their separate responsibilities.
The 12-domain enterprise AI agent audit framework
Use this table to set the audit universe, assign evidence requests, and write procedures. Each row needs a defined population, a named owner, a testable control, a source-of-truth record, and an explicit failure condition before fieldwork starts.
| Audit domain | Auditor's question | Accountable owner | Control to test | Evidence required | Failure signal | Where in KLA / deeper guide |
|---|---|---|---|---|---|---|
| 1. Inventory and scope | Which agents, Releases, environments, Processes, decisions, and dependencies are in the population? | Business owner | Inventory reconciliation and boundary approval | Agent inventory, Process map, data flow, dependency register, population counts | Unknown agent, missing environment, unreconciled population | Agent Registry; compliance guide |
| 2. Responsibility and accountability | Who owns outcomes, control operation, risk acceptance, and remediation? | Business owner | Named ownership and approval authority | Responsibility matrix, role charters, sign-offs, issue owners | Shared accountability, vacant role, owner without authority | Agent Registry; Control Mapping |
| 3. Identity and delegation | Can every action be bound to an agent, sponsoring principal, and delegation? | Identity owner | Unique identity and scoped delegation | Identity records, token claims, delegation chain, session scope | Shared credential, orphan identity, unbound principal | Agent Registry; permissions guide |
| 4. Permissions, tools, and data boundaries | Could the agent read or change resources beyond its approved purpose? | Application owner | Least-privilege policy at action time | Effective grants, Tool Catalog entry, Data Boundaries, policy verdicts | Broad scope, direct bypass, unregistered tool, toxic grant combination | Tool Catalog; Data Boundaries; KLA Policy Engine |
| 5. Risk classification and pre-production assurance | Was the use classified and tested against its actual impact and context? | Risk owner | Documented classification and release gate | Impact assessment, threat model, evaluation plan, acceptance thresholds | Unsupported classification, missing test, waived failed threshold | Control Mapping; Assurance Center |
| 6. Runtime policy enforcement | Did the approved policy evaluate before each consequential action? | Control owner | Inline allow, deny, or escalate decision | Policy version, matched rules, Decision Request, action receipt | Action precedes verdict, stale policy, enforcement bypass | Policy Builder; KLA Policy Engine; Audit Trail; govern-an-agent guide |
| 7. Human approval and escalation | Did an authorized reviewer receive enough evidence and exercise judgment? | Operations owner | Risk-based approval and escalation | Decision Request, authority snapshot, evidence shown, rationale, timestamps | Rubber stamp, expired authority, late approval, missing rationale | Decision Desk; accountable autonomy |
| 8. Execution lineage and business outcomes | Can the auditor trace intent through tool effects and final outcome? | Process owner | End-to-end correlation and outcome reconciliation | Lineage Record, Journey, tool calls, before/after state, outcome record | Broken correlation, unrecorded side effect, outcome mismatch | Lineage Explorer; audit trails guide |
| 9. Continuous assurance and change management | Did changes trigger reassessment, monitoring, and controlled Rollout? | Engineering owner | Release approval, drift detection, change-triggered testing | Release diff, test results, Rollout record, Assurance Alerts, remediation | Unapproved change, silent drift, unresolved failed control | Agents; Assurance Center; monitoring guide |
| 10. Incident response, revocation, and rollback | Could the organization contain the agent and reverse affected actions? | Incident owner | Kill, revoke, contain, notify, and Rollback procedure | Incident timeline, credential revocation, Rollback, affected-case list | Continued execution, incomplete scope, failed Rollback | Security Center; Agents; Evidence Room |
| 11. Evidence integrity, retention, and Independent Verification | Is the evidence complete, tamper-evident, retained, and independently testable? | Records owner | Population reconciliation, sealing, retention, verifier test | Manifest, hashes, signatures, chain of custody, retention policy, legal hold | Hash failure, missing record, mutable source, expired retention | Evidence Room; Sealed Evidence Bundle; Control Pack; tamper-evident evidence; sample bundle |
| 12. Multi-agent and third-party dependencies | Are delegated agents, models, tools, protocols, and vendors inside the audit boundary? | Service owner | Dependency approval and authenticated handoff | Supplier inventory, contracts, versions, inter-agent messages, control reports | Opaque sub-agent, unauthenticated message, unsupported component | Agent Registry; Tool Catalog; OWASP crosswalk |
Define the system boundary before sampling activity
Start with the business outcome and trace inward. The boundary includes every component that can influence an action or its proof: agent configuration, model, prompt and orchestration versions, memory, retrieval sources, user and service identities, delegation, policy, tools, downstream systems, human reviewers, monitoring, incident processes, and evidence stores. Include sub-agents and third parties whenever their output can change the final decision, available authority, or record completeness.
Build a population that can be reconciled. The NIST AI RMF 1.0 is a final voluntary framework for AI broadly, and GOVERN 1.6 supports maintaining an AI-system inventory for risk management. For audit fieldwork, extend that inventory to Releases, Rollouts, decision types, environments, tools, data sources, owners, risk tiers, incident history, and evidence locations.
Write the boundary statement as an audit artifact and obtain business-owner approval. A useful statement names the audited Process, start and end events, included agents and Releases, production period, decision population, jurisdictions, excluded components with reasons, upstream data, downstream actions, human roles, and every external dependency. Scope changes during fieldwork require a dated amendment and an impact assessment for the sample.
- Population keys: agent ID, Release ID, environment, Process ID, decision type, outcome, risk tier, and date range.
- Authority boundary: sponsoring principal, agent identity, delegated scopes, permitted tools, resource limits, purpose, duration, and approval thresholds.
- Execution boundary: every model call, tool call, inter-agent handoff, policy verdict, Decision Request, state change, notification, and business outcome.
- Evidence boundary: system of record for each artifact, retention class, sealing method, verifier, legal hold, and known collection gap.
Set accountability and separate four assurance disciplines
Assign one accountable business owner for the agent system and its outcomes. Name technical owners for the agent, model, integrations, identity, data, and evidence infrastructure; name control owners for policy, approval, monitoring, incident response, and retention. Each owner needs authority to stop a Rollout, accept a defined risk within delegated limits, fund remediation, and answer an exception.
The IIA Three Lines Model assigns risk ownership and management to first-line roles, expertise and challenge to second-line roles, and independent and objective assurance to internal audit. Apply those roles to the agent audit without turning them into three fixed departments. Internal audit preserves independence by avoiding control ownership and management approval decisions for the system it later audits.
Four disciplines contribute different evidence. Their work can be reused when scope, period, criteria, competence, and independence are documented. Their conclusions remain distinct.
| Discipline | Primary objective | Procedure | Evidence | Conclusion |
|---|---|---|---|---|
| Model evaluation | Measure behavior against defined tasks and risk criteria | Benchmark, scenario, red-team, subgroup, and regression tests | Dataset/version, method, thresholds, results, limitations | Performance for tested conditions |
| Security testing | Find exploitable paths across goals, tools, identity, memory, code, and dependencies | Threat modeling, adversarial testing, configuration review, exploit validation | Threat model, test cases, exploit trace, severity, remediation retest | Security exposure for tested scope |
| Compliance audit | Assess defined legal, regulatory, contractual, and policy criteria | Design test, operating-effectiveness sample, evidence inspection, reperformance | Criteria matrix, population, sample, workpapers, exceptions, management response | Conformity or exception against stated criteria |
| Operational assurance | Verify controls continue to operate through production change | Continuous signals, threshold alerts, reconciliations, targeted review, remediation tracking | Control events, Assurance Alerts, owner review, issue closure evidence | Current control health and unresolved exposure |
Run design-time, release-time, runtime, and periodic procedures
Treat the audit program as a lifecycle. The final voluntary NIST Generative AI Profile suggests retaining test, evaluation, validation, and verification history, using measurable release criteria, documenting approval, conducting ongoing evaluation, and defining deactivation procedures. Apply those practices where the agent uses generative AI, then add agent-specific identity, delegation, tool, policy, approval, and multi-agent procedures.
Singapore's IMDA Model AI Governance Framework for Agentic AI v1.5 is published voluntary living guidance. It supports defining operating bounds and permission policies before deployment, evaluating components and end-to-end behavior, monitoring after deployment, handling incidents, and reassessing changes to models, tools, permissions, and Processes.
| Stage | Required procedures | Evidence population | Pass criterion |
|---|---|---|---|
| Design time | Boundary, purpose, ownership, risk classification, impact assessment, threat model, identity model, tool and Data Boundaries, approval design, evidence schema | Design records and approved control specification | Every material risk maps to an owner, control, test, and evidence field |
| Release time | Regression and adversarial tests, policy Simulation, permission review, dependency review, evidence completeness test, Rollback rehearsal, approval | Release candidate, test suite, exceptions, sign-offs | Thresholds pass; accepted exceptions are authorized, dated, and bounded |
| Runtime | Inline policy enforcement, Decision Routing, identity binding, evidence capture, anomaly detection, rate and value limits, containment | All production actions and control events | Consequential actions carry a prior verdict and complete Lineage Record |
| Periodic | Population reconciliation, risk-based sample, access recertification, approval-quality review, drift and outcome analysis, incident follow-up, retention and verifier tests | Defined period plus all mandatory exception strata | Exceptions are quantified, owned, remediated, and reflected in the audit conclusion |
Select samples by risk, decision type, anomaly, approval path, and system change
Establish completeness before choosing cases. Reconcile originating business events, agent runs, policy decisions, Decision Requests, downstream effects, and sealed evidence records. Differences become exceptions or a scope limitation; they cannot disappear through sample selection.
Use a reproducible, risk-based sample with a random baseline. Record the population query, extraction time, source systems, filters, random seed, selection logic, replacements, and reviewer. Preserve the frozen population with hashes so a second reviewer can regenerate the same sample.
- Risk: include the highest impact, value, privilege, sensitivity, irreversibility, and affected-person cases.
- Decision type: cover each material allow, deny, escalate, override, rollback, and no-action outcome.
- Anomaly: include control bypasses, policy denials followed by execution, repeated retries, unusual tool sequences, drift alerts, latency spikes, and outlier outcomes.
- Approval path: include autonomous actions, ordinary approvals, escalations, overrides, break-glass use, expired requests, and reviewer reassignment.
- System change: include the first and last cases around model, prompt, policy, permission, tool, data, orchestrator, Release, and Rollout changes.
- Random baseline: select from the remaining population to detect ordinary failures that risk filters may miss.
Worked audit: one regulated loan-decision agent action
Assume an EU retail bank uses an agent to assemble application data, call a credit-risk model, apply lending policy, route borderline cases to an underwriter, and write the approve or decline outcome. An AI system intended to evaluate a natural person's creditworthiness or establish a credit score is within Annex III point 5(b) of the EU AI Act and is presumed high-risk under Article 6(2), subject to the specific Article 6(3) rules; profiling in an Annex III use remains high-risk. The audit records the bank's fact-specific deployer role and the model vendor's provider role under Article 3, then tests each party's applicable controls.
For a high-risk system, Article 12 of the EU AI Act requires technical capability for automatic event logging over the system lifetime to support traceability, post-market monitoring, and investigation. Article 14 requires effective human-oversight capability proportionate to risk, autonomy, and context, while Article 26(2) requires the deployer to assign oversight to people with the necessary competence, training, authority, and support. The walkthrough converts those conditional duties into evidence tests for one decision.
| Step | Expected control | Artifact | Auditor procedure | Failure condition |
|---|---|---|---|---|
| 1. Establish the case | Application enters the approved lending Process and receives a unique Journey ID | Application event, Journey ID, purpose code, decision type, risk tier | Trace the business event into the agent population and sealed record | Missing case, duplicate ID, purpose mismatch |
| 2. Bind identity and authority | Agent identity, sponsoring principal, session, and delegation are current and scoped | Identity claims, delegation record, authority snapshot, expiry | Reperform effective authority at the event timestamp | Shared identity, expired grant, excess scope |
| 3. Retrieve permitted data | Data Boundaries limit sources, records, fields, geography, and purpose | Source references, query hash, boundary ID/version, redaction record | Compare accessed records with the approved boundary and source logs | Unapproved source, excess field, missing provenance |
| 4. Invoke model and tools | Approved model, prompt, orchestrator, and tool versions execute with bounded inputs | Release IDs, input/output hashes, Tool Catalog versions, invocation receipts | Resolve every version and compare the call sequence with the approved Release | Unapproved version, hidden tool, mutable input |
| 5. Enforce policy | KLA Policy Engine evaluates the action before execution | Policy ID/version, matched rules, allow/deny/escalate verdict, timestamp | Reperform the policy decision with the recorded inputs and version | Verdict follows action, rule mismatch, bypass |
| 6. Obtain human decision | Borderline or exception case routes to an authorized underwriter | Decision Request, evidence shown, reviewer authority, rationale, decision time | Inspect evidence sufficiency and independently validate reviewer authority | Rubber stamp, missing rationale, unauthorized reviewer |
| 7. Commit the outcome | Approved tool action matches the policy and human decision | Tool request/response, before/after state hashes, loan-system reference | Trace the final write in the bank system and compare amount, terms, and status | Outcome differs, extra side effect, missing receipt |
| 8. Notify and preserve remedy | Required notice, review route, escalation, and correction path remain linked to the case | Notice record, reason codes, appeal or manual-review event, correction record | Inspect delivery and trace any later challenge through resolution | Undelivered notice, broken review path, unresolved correction |
| 9. Seal and verify evidence | Evidence Room seals the complete record and verifier tests integrity | Sealed Evidence Bundle, manifest, hashes, signature, chain of custody | Recalculate hashes, validate signature, and reconcile manifest to source events | Hash failure, omitted artifact, unknown key |
| 10. Replay the decision | Lineage Explorer resolves versions, inputs, policy, approval, and effects | Replay record, version archive, deterministic results or documented tolerance | Reperform policy and tool sequence in a controlled environment | Missing version, unexplained divergence, unsafe live side effect |
Responsibility matrix for control ownership and independent assurance
Keep the matrix compact and decision-specific. One accountable business owner signs the scope, risk acceptance, and remediation plan. Responsible technical and control owners operate the controls. Risk, compliance, security, privacy, legal, and model-risk functions challenge within their mandates. Internal audit sets its own scope, performs independent procedures, and reports conclusions to the appropriate governing body.
| Activity | Accountable | Responsible | Consulted / challenged by | Evidence of accountability |
|---|---|---|---|---|
| Approve purpose, risk appetite, and production use | Business owner | Product and Process owners | Risk, compliance, legal, security | Signed use approval and conditions |
| Design agent, model, tools, and data controls | Technical owner | Engineering, model, identity, data, platform owners | Risk, security, privacy, control owners | Approved design and control specification |
| Operate policy, approval, monitoring, and incidents | Operations owner | Control owners and on-call teams | Risk, compliance, security operations | Control events, review logs, incident records |
| Approve Release, Rollout, exception, and Rollback | Business owner | Release manager and technical owner | Control, risk, security, model-validation owners | Decision record with test results and conditions |
| Retain and verify evidence | Records owner | Evidence platform and source-system owners | Legal, privacy, internal controls | Retention schedule, verifier results, legal holds |
| Provide independent audit conclusion | Chief audit executive or audit committee delegate | Internal audit engagement team | Subject-matter specialists with independence safeguards | Approved audit plan, workpapers, report, follow-up |
Minimum evidence schema for AI agent audit logs, replay, and action proof
A replayable trail needs stable identifiers, versioned context, control results, human authority, business effects, and integrity metadata. The AI agent audit-trails guide explains the evidence layers, the tamper-evident evidence methodology covers integrity, and the Evidence Room sample shows the export shape.
Store sensitive values according to approved privacy and security controls. The audit schema can retain a protected value, a stable reference, or a hash according to the field purpose. The auditor tests whether the representation supports the stated assertion and can be resolved under authorized review.
| Field group | Minimum fields | Audit test |
|---|---|---|
| Record and correlation | `record_id`, `occurred_at`, `environment`, `tenant_id`, `process_id`, `journey_id`, `correlation_id` | Uniqueness, timestamp ordering, population reconciliation |
| Agent and release | `agent_id`, `agent_release_id`, `model_id`, `model_version`, `orchestrator_version`, `prompt_template_version` | Resolve every version to an approved immutable artifact |
| Principal and delegation | `agent_identity_id`, `sponsoring_principal_id`, `delegated_by`, `session_id`, `authority_snapshot_id`, `expires_at` | Reperform effective authority at event time |
| Purpose and risk | `purpose_code`, `decision_type`, `risk_tier`, `regulatory_classification`, `classification_basis_version` | Compare purpose and classification with approved scope |
| Data provenance | `input_reference[]`, `source_hash[]`, `retrieval_query_hash`, `data_boundary_id`, `redaction_profile_id` | Trace every material input to an allowed source and boundary |
| Tool action | `tool_id`, `tool_version`, `action`, `resource_scope`, `requested_args_hash`, `response_hash`, `effect_id` | Match requested authority, actual call, response, and side effect |
| Policy decision | `policy_id`, `policy_version`, `policy_decision`, `matched_rule_ids[]`, `exception_id`, `evaluated_at` | Reperform verdict and confirm it precedes execution |
| Human decision | `decision_request_id`, `reviewer_id`, `reviewer_role`, `presented_evidence_hash`, `decision`, `rationale`, `decided_at` | Validate authority, evidence presented, timing, and rationale |
| Outcome and recovery | `outcome`, `before_state_hash`, `after_state_hash`, `external_reference`, `incident_id`, `rollback_id`, `revocation_event_id` | Reconcile recorded outcome with the business system and recovery history |
| Integrity and retention | `evidence_hash`, `previous_record_hash`, `bundle_manifest_hash`, `signature_key_id`, `sealed_at`, `retention_class`, `legal_hold_id` | Recalculate hashes, validate signature and chain, inspect retention and hold |
Test access rights, delegation, and multi-agent dependencies
NIST's AI Agent Standards Initiative is an active roadmap covering standards, protocols, identity, authorization, security, interoperability, and evaluation. It is not a final standard. The related NCCoE identity and authorization concept paper remains a draft and studies non-human identity, scoped delegation, least privilege, action logging, provenance, non-repudiation, and tamper-evident records. Use it to sharpen audit questions and label the criteria as organization-defined controls.
Test the complete authority chain: the sponsoring human or organizational principal, agent identity, delegated scope, purpose, time window, tool capability, resource and action limits, approval thresholds, exception path, revocation path, and the authority actually observed at execution. The AI agent permissions guide provides the deeper control model.
For multi-agent systems, authenticate and authorize each handoff, preserve the sending and receiving identities, validate message integrity and semantics, propagate purpose and constraints, limit sub-agent delegation, and reconcile the final outcome to every contributing component. The OWASP Top 10 for Agentic Applications 2026 is published practitioner guidance and a security taxonomy; it covers identity and privilege abuse, tool misuse, memory, inter-agent communication, cascading failure, and rogue agents. It is not a formal standard or certification.
- Identity test: every non-human actor is unique, active, owned, and distinguishable from people and peer agents.
- Delegation test: delegated authority is linked to an approved principal, purpose, scope, duration, and revocation event.
- Effective-access test: source grants, policy constraints, tool enforcement, and observed actions reconcile at the event timestamp.
- Third-party test: contracts, service inventory, versions, access paths, incident duties, evidence availability, and termination controls cover the dependency.
- Multi-agent test: each message carries authenticated sender, intended recipient, integrity evidence, context limits, and correlation to the final Journey.
Set audit cadence and Continuous Assurance by trigger
NIST's final report Challenges to the Monitoring of Deployed AI Systems explains that post-deployment monitoring can detect reliability problems, drift, and unintended consequences, while monitoring goals and methods depend on the system, context, available signals, and actors. It does not prescribe one cadence, minimum schema, or universal threshold. Set cadence from impact, autonomy, volume, change rate, incident history, detectability, legal duties, and evidence quality.
Continuous Assurance supplies full-population control signals to management and targeted populations to second-line and internal-audit teams. Internal audit still validates source completeness, control design, alert logic, owner review, remediation, and independence before relying on those signals. The post-market monitoring guide provides the deeper monitoring structure.
| Trigger or interval | Procedure | Primary owner | Audit evidence |
|---|---|---|---|
| Before first production use | Full 12-domain design and readiness review | Business and technical owners | Approved boundary, controls, tests, evidence, Rollback |
| Every material Release or boundary change | Regression, policy Simulation, permission and dependency review, evidence test | Release owner | Release diff, test results, approval, Rollout conditions |
| Continuous runtime | Policy, approval, identity, anomaly, outcome, and evidence-integrity signals | Control owners | Control events, Assurance Alerts, incident linkage |
| Weekly or monthly operations review | Exceptions, overrides, denial patterns, unresolved alerts, Rollbacks | Operations owner | Review record, decisions, Remediation Plans |
| Quarterly risk review | Access recertification, outcome analysis, sample testing, supplier changes | Business and risk owners | Recertification, sample workpapers, risk acceptance |
| Annual or risk-based audit cycle | Independent scope, design and operating-effectiveness testing, follow-up | Internal audit | Audit plan, workpapers, report, verified closure |
| Incident or material control failure | Containment, full affected population, root cause, Rollback, control retest | Incident owner | Incident file, affected-case list, recovery and retest evidence |
Apply regulatory and standards criteria with current status
No final, agent-specific, end-to-end auditability standard was verified as of the 14 July 2026 source review. NIST AI RMF 1.0 is a final voluntary framework for AI risk management across technologies and sectors. The NIST AI Agent Standards Initiative is a roadmap, and its identity and authorization work remains a draft concept paper. Audit teams therefore need a documented criteria stack tailored to the system and engagement.
The IMDA Model AI Governance Framework for Agentic AI v1.5 is published voluntary living guidance. The OWASP Agentic Top 10 is published practitioner security guidance and a risk taxonomy. Both provide useful agent-specific criteria, and neither is a certification standard.
ISO/IEC 42001:2023 specifies requirements for an organizational AI management system. A certification audit can assess conformity of the AIMS within its declared scope. The certificate does not certify an individual model, agent, decision, output, or dataset as safe, accurate, fair, secure, ethical, or lawful; verify the certification body, accreditation, sites, exclusions, validity, and statement of applicability.
The EU AI Act applies according to operator role, intended purpose, and risk classification. Provider and deployer are distinct, fact-dependent roles under Article 3, and Article 25 can shift provider responsibility after rebranding, substantial modification, or a purpose change that makes a system high-risk. AI agents are not automatically high-risk; apply Article 6 and Annex I or III to the actual system, purpose, and facts.
For high-risk systems, Article 12 requires technical capability for automatic event logging over the system lifetime. Article 19 requires providers to retain automatically generated logs under their control for at least six months unless another applicable law provides otherwise. Article 26(6) gives deployers the same six-month rule for logs under their control, while Article 26 also covers following instructions, assigning competent and authorized oversight, monitoring operation, and acting when risks or serious incidents arise. These duties do not create a universal six-month retention rule for every agent or every log.
The EU co-legislators adopted the Digital Omnibus on AI in June 2026. The final adopted text sets 2 December 2027 for Article 6(2)/Annex III stand-alone high-risk systems and 2 August 2028 for Article 6(1)/Annex I product-embedded systems. As of the 14 July 2026 source check, the Council final-adoption notice said Official Journal publication would occur shortly and entry into force would follow on the third day after publication, so the amendment had been finally adopted and was not yet verified in force. Recheck EUR-Lex before relying on the dates in a legal conclusion.
Handle incomplete evidence and qualify the audit conclusion
Evidence sufficiency requires relevance, reliability, completeness, integrity, timeliness, and traceability to the population and assertion. Document which party produced each artifact, which system is authoritative, how it was extracted, whether it can be altered, how the sample ties to the population, and whether an independent reviewer can repeat the procedure.
Classify every gap before concluding: a control failure, a missing record, an integrity failure, an unavailable third-party artifact, a retention expiry, a population limitation, or an audit-scope exclusion. Perform alternative procedures where they address the same assertion. Examples include source-system reperformance, downstream-state reconciliation, independent signature validation, confirmation from an external party, or testing a larger affected population.
State the limitation in the report with the affected domain, period, population, assertion, attempted alternatives, residual uncertainty, risk, responsible owner, remediation, and due date. A qualified conclusion identifies the reliable areas and the exact boundary around unsupported assurance. A disclaimer or management representation cannot replace missing operating evidence.
| Evidence condition | Alternative procedure | Conclusion treatment |
|---|---|---|
| Population does not reconcile | Rebuild from originating and downstream systems; test all unmatched cases | Scope limitation over completeness until reconciled |
| Policy or Release version unavailable | Inspect archive, deployment artifact, signed manifest, and source history | No reperformance conclusion for affected cases if version remains unresolved |
| Approval rationale or authority missing | Inspect identity records, Decision Desk history, and reviewer confirmation | Control exception; confirmation alone does not prove contemporaneous judgment |
| Hash, signature, or chain validation fails | Recalculate from authoritative artifacts and inspect key and custody history | Integrity exception across the affected bundle or chain segment |
| Third-party sub-agent evidence unavailable | Inspect contract, independent report, gateway records, input/output receipts, and outcome reconciliation | Qualified conclusion over the opaque dependency and affected assertions |
| Retention expired before audit | Inspect approved schedule, legal holds, surviving source records, and downstream outcomes | Period limitation plus retention-control finding where criteria required preservation |
Frequently Asked Questions
What does an AI agent audit cover?
It covers the full production system around the agent: inventory, accountable owners, identities, delegation, permissions, data and tools, risk classification, release tests, runtime policy, human decisions, Execution Lineage, outcomes, changes, incidents, evidence integrity, retention, and third parties. The audit tests both control design and operating effectiveness over a reconciled population.
How can I audit and replay AI agent decisions using enterprise data logs?
Correlate business events, agent and model versions, data references, tool calls, policy verdicts, approvals, and downstream state changes under stable IDs. Freeze the versions and input references, verify the evidence manifest, reperform the policy decision, and replay tool effects in a controlled environment; the Lineage Explorer workflow shows the governed sequence.
How do I create audit trails for AI agent actions?
Capture records synchronously at identity, retrieval, policy, approval, tool, outcome, incident, and Rollback events. Seal them into a manifest with hashes, signatures, chain-of-custody data, and retention metadata, then reconcile every action to the originating and downstream systems; see the AI agent audit-trails guide.
What audit trails do I need for AI agent compliance under the EU AI Act?
The requirement depends on role and classification. For high-risk systems, Article 12 of the EU AI Act requires technical capability for automatic event logs over the system lifetime; Articles 19 and 26(6) require providers and deployers respectively to retain automatically generated logs under their control for at least six months unless another applicable law provides otherwise. This rule does not apply universally to every AI agent or every log.
How do I audit and certify AI agent access rights?
Audit the sponsoring principal, agent identity, delegation, effective grants, tool and resource boundaries, purpose, duration, approval conditions, observed use, revocation, and recertification evidence. NIST's identity and authorization paper remains a draft, and ISO/IEC 42001 certifies a scoped organizational management system through a certification body; neither provides a universal certificate for an individual agent's access rights.
How often should an AI agent system be audited?
Audit before first production use, after material changes, on a risk-based periodic cycle, and after incidents or material control failures. Continuous Assurance should monitor the full population between audits; the final NIST monitoring report confirms that monitoring depends on system context and does not prescribe one universal cadence.
What happens when AI agent audit evidence is incomplete?
Classify the gap, quantify the affected population, perform alternative procedures, and record the residual uncertainty. The report should qualify the affected assertion, period, or dependency and assign remediation; management representation does not replace contemporaneous operating evidence.
Key Takeaways
A defensible AI agent audit starts with a complete boundary and population, assigns one accountable business owner, tests controls across the lifecycle, samples consequential and anomalous actions, replays the Execution Lineage, and qualifies every conclusion that lacks sufficient evidence. Use the Agent Audit Readiness Assessment to score the 12 domains, then inspect the sample Sealed Evidence Bundle to compare your records with an independently verifiable evidence package.