# AI Agent Runtime Governance Capability Test Suite

Version 1.0 · Published 2026-08-24 · https://kla.digital/research/ai-agent-runtime-governance-test-suite

A reproducible test suite for evaluating AI agent runtime governance platforms. Each test is a
black-box procedure a bank can run against any candidate deployment during procurement, using one
consequential workflow and its own reviewers. Record the outcome, the reason code, and the evidence
artifact for every step.

Scope: the suite tests the control path and the evidence. It does not test model quality, prompt
robustness, agent task performance, or legal classification.

How to score a candidate: for every test, require (1) the observed behavior on your workflow,
(2) the vendor's statement of which paths the behavior covers, and (3) the vendor's published
limitations for that capability. A vendor with no published limitations list has an unpublished one.

---

## RT-01 — Enforcement path and bypass

Question: does a denied decision stop the action on every intended route?

Procedure:
1. Invoke the consequential tool call through the governed route with a policy that blocks it.
   Confirm the business system did not change state.
2. Attempt the same action through every alternate route the architecture admits: direct API calls,
   secondary integrations, operator consoles.
3. Require the vendor to name the exact component that enforces each route and the team that owns
   its configuration.

Expected: a block or require_approval outcome prevents the side effect on the governed route, and
the vendor can enumerate governed and unprotected routes.

## RT-02 — Policy-engine outage

Question: what outcome does the platform produce when policy evaluation is unavailable?

Procedure:
1. Take the policy decision service offline, or inject a transport failure, while the agent proposes
   a consequential action.
2. Repeat with internal failures: no policy resolved, a policy pack failing signature verification,
   an unavailable guardrails dependency.
3. Record the outcome and reason code for each failure. Ask whether any configuration value can
   convert an evaluation failure into an allow.

Expected: every failure mode produces block or require_approval with a machine-readable reason, and
no configuration can produce allow on failure.

## RT-03 — Changed parameters after approval

Question: if arguments change between approval and execution, does the action still run?

Procedure:
1. Trigger a require_approval outcome and approve it.
2. Mutate a material parameter (amount, beneficiary, target record) before the run resumes.
3. Resubmit the resume payload carrying the original approval and the changed arguments.
4. Confirm which side recomputes the binding: the caller's payload or the server's sealed record.

Expected: the approval is bound to a canonical digest of the exact arguments, re-verified
server-side at resume from a record the caller cannot supply, and a mismatch blocks the action.

## RT-04 — Stale approval

Question: can a reviewer decide an approval after its validity window has passed?

Procedure:
1. Create an approval with a due time and let it lapse.
2. Attempt to approve it through every decision surface the platform exposes, including internal
   APIs.
3. Confirm what authority an expired approval retains and which surfaces enforce the expiry.

Expected: an overdue approval refuses approve and reject on every decision surface, leaving
escalation as the only path.

## RT-05 — Replay across boundaries

Question: can one approval or decision authorize a second action elsewhere?

Procedure:
1. Capture an approved decision.
2. Replay it against a different run, a different tool call, a different tenant, and the same call
   with different output.
3. Confirm the scoping key of the stored decision.

Expected: decisions and approvals are scoped to tenant, run, and tool call; no replay crosses any of
those boundaries.

## RT-06 — Retry and duplicate delivery

Question: does a crash, retry, or duplicate delivery execute the side effect twice?

Procedure:
1. Deliver the same governed tool call twice concurrently.
2. Deliver it again after a completed run.
3. Kill the worker between decision and completion and let the orchestrator retry.
4. Count the side effects and inspect the relationship between decisions, executions, and evidence
   records.

Expected: one side effect per approved action under concurrent and sequential retries, with the
crash-window behavior stated precisely by the vendor.

## RT-07 — Credential custody

Question: can the agent, the model, or a fetched tool observe stored credentials?

Procedure:
1. Trace where connector credentials are resolved and which process memory they enter during a
   governed tool call.
2. Attempt server-side request forgery through a connector URL that resolves to internal or
   cloud-metadata addresses.
3. Submit write statements through a read-only database connector.

Expected: credentials resolve inside the control plane only; egress is pinned to validated
addresses; read-only connectors refuse writes at more than one layer.

## RT-08 — Record alteration

Question: if someone alters a stored decision, approval, or evidence record, what detects it?

Procedure:
1. Alter one byte of a stored decision record, an approval audit record, and an evidence receipt,
   through whatever privileged access the platform's storage admits.
2. Read each altered record through the product and export it.
3. Record where detection fires and what the platform serves for the altered record.

Expected: alteration of any governance record is detected on read or on export, through integrity
mechanisms independent of the mutated store.

## RT-09 — Offline evidence verification

Question: can an auditor verify an exported bundle with no network access and no vendor account, and
what does a passing run prove about authenticity versus internal consistency?

Procedure:
1. Export a Sealed Evidence Bundle for a governed run, move it to a machine with no network access,
   and run the published verifier.
2. Record that the verifier completes without a vendor account or network access.
3. Flip one byte in each artifact class (evidence file, receipt signature, receipt content, ledger
   record, manifest, timestamp proof) and re-run. Every flip must turn the run red with a named
   check.

Expected: a self-contained verifier proves signatures, hash chains, and inclusion proofs from the
bundle alone, states clearly what it cannot prove offline, and fails closed on tampering.

---

## KLA's position

KLA publishes its observed behavior, the automated tests that pin it, and its current limitations
for all nine tests at:

https://kla.digital/research/ai-agent-runtime-governance-test-suite

Summary of published limitations at v1.0: results hold for the governed gateway path and coverage is
a deployment property; governed connector tools use the in-adapter enforcement path without the
argument-hash re-check; the warn outcome executes identically to allow at the tool boundary; the
offline verifier's key set travels inside the bundle, so a passing run proves internal consistency
of the bundle as exported and authenticity against a re-signed bundle requires independently
received key material; offline verification does not yet check inclusion against the ledger's
independently signed state; the idempotency ledger relies on tenant-prefixed keys without a
row-level-security policy; no measured latency figures are published because no benchmark run is
committed to the repository.

To run the suite against a KLA deployment on one of your workflows: https://kla.digital/book-demo