When regulators audit your AI systems, they're not just checking for documentation - they want evidence that your governance controls actually work. Here are the 10 questions you should be ready to answer.
1. How do you classify your AI systems by risk?
Auditors will ask for your AI system inventory and classification methodology. They want to see that you've systematically evaluated each system against regulatory risk frameworks.
- Complete inventory of AI/ML systems in production
- Risk classification for each system with documented rationale
- Process for reassessing classification when systems change
2. Who approved this AI system for deployment?
Before any high-risk system goes live, there should be documented approval from appropriate stakeholders - not just engineering sign-off.
3. How do humans oversee AI decisions?
This is where many organizations struggle. Auditors want evidence of effective human oversight, not just policies stating it exists.
- Documented oversight procedures
- Evidence that humans can and do intervene
- Records of human approvals and overrides
4. Can you show me the decision trail for this specific case?
Auditors may pick individual cases and ask you to trace the decision from input to output, including any human reviews. Your audit trail must be complete and accessible.
5. How do you detect and handle AI failures or errors?
What happens when your AI system makes a mistake? Auditors want to see monitoring, alerting, and incident response procedures.
6. What training data was used and how was it governed?
Data governance is critical. Be prepared to explain data sources, quality controls, bias assessments, and privacy compliance.
7. How do you prevent and detect model drift?
AI systems degrade over time. Show your monitoring approach for detecting when models drift from expected behavior.
8. Who can access and modify the AI system?
Access controls, change management, and audit logs for system modifications are baseline expectations.
9. How long do you retain AI decision records?
Retention policies must align with regulatory requirements - often 5-7+ years for regulated industries.
10. Can you export evidence for independent verification?
Modern auditors expect to verify evidence independently, not just trust internal reports. Your evidence exports should include integrity verification (checksums, signatures).
Frequently Asked Questions
How far in advance should I prepare for an AI audit?
Start now. The best time to prepare was when you deployed the system. The second best time is today. Mock audits should happen at least quarterly.
What evidence format do auditors prefer?
Auditors want structured, verifiable evidence - not raw logs. Evidence packs with manifests, checksums, and clear mapping to regulatory requirements make audits faster and more successful.
Key Takeaways
The common thread across all these questions is evidence. Policies and procedures matter, but auditors ultimately want proof that your controls work in practice. Build your evidence collection into your AI workflows from day one.