# RFP Checklist — KLA vs LangSmith (Regulated Agent Workflows)

Last updated: 2025-12-17 · Version v1.0  
Not legal advice.

## How to use (5 minutes)
1. Mark which **deliverables you must produce** (Annex IV, oversight SOP, monitoring plan, etc.).
2. For each vendor, ask for a **sample export** of the deliverable (not a slide).
3. Run a short **evidence export drill** (below) before procurement.

## Audit deliverables (minimum viable)
- Annex IV-style technical documentation export mapping (fields → evidence)
- Post-market monitoring plan + risk-tiered sampling policy
- Human oversight procedure (queues, escalation, overrides) + decision records
- Audit log retention policy + integrity verification mechanics
- Evidence pack export bundle (manifest + checksums; redaction rules documented)

## Questions to ask any vendor
### Governance controls
- Can you enforce **policy-as-code checkpoints** (block/review/allow) for high-risk workflow actions?
- Do you support **role-aware approval queues** and escalation (with captured reviewer context)?
- How do you handle **overrides** (required rationale, attachments, two-person rule)?

### Monitoring + sampling
- Can you define a **sampling policy** (risk-tiered rates, burst rules, reviewer rubrics)?
- Do you track **near-misses** (blocked/nearly blocked steps) as a control signal?
- Can you show monitoring outcomes linked to the exact workflow versions in production?

### Proof + exports
- Can an auditor independently verify an export (manifest + checksums)?
- What is the default retention posture (e.g., “7+ years”) and how is integrity proven?
- Can you export an evidence pack without leaking sensitive data (redaction + access controls)?

## Where LangSmith typically fits
- Strengths: developer-first tracing and evaluation loops for LangChain/LangGraph apps.
- Common gap to close for audits: approvals/overrides, policy enforcement evidence, and verifiable evidence-pack exports mapped to Annex IV deliverables.

## Acceptance criteria for purchase
- You can produce (and export) the artifacts you must submit or present: Annex IV pack, monitoring plan, oversight records.
- The system captures **who approved what** under **which policy version**, with reviewer context, timestamps, and trace links.
- Evidence exports include integrity proofs that a third party can verify.

## 30-minute evidence export drill
1. Pick one high-risk workflow action (e.g., eligibility decision, account closure, SAR recommendation).
2. Trigger an approval + an override (with rationale).
3. Export the evidence pack and verify you can answer: who/what/when/why + policy version + sampling outcome + integrity checks.

Links:
- Compare page: `/compare/kla-vs-langsmith`
- Evidence pack checklist: `/resources/evidence-pack-checklist`
- Sample Evidence Room export: `/downloads/evidence-room-sample.pdf`