EU AI Act Compliance for Healthcare

Diagnostic AI: Typically high-risk and medical device

AI systems used for diagnosis, screening, or detection of disease are typically medical devices under MDR (if they provide diagnostic information used in patient care) and high-risk under EU AI Act Annex III (category 6(a) covers AI intended for use as medical devices).

• Radiology AI for detecting tumors or abnormalities
• Pathology AI for analyzing tissue samples
• Screening tools for disease risk assessment
• Diagnostic support for specific conditions

Clinical decision support: Depends on intended purpose

Clinical decision support (CDS) systems occupy a spectrum from minimal risk to high-risk depending on their design and intended purpose.

Likely high-risk: CDS that provides treatment recommendations, systems that generate differential diagnoses, AI that recommends medication dosing, tools that guide surgical planning.

Potentially NOT high-risk: CDS that provides reference information, systems that surface relevant literature without recommendations, tools that assist with documentation without clinical guidance.

Administrative AI: Usually minimal risk

Administrative and operational AI typically falls outside high-risk categories: scheduling optimization (not high-risk unless it affects clinical priority), billing and coding (administrative function), resource allocation (may be high-risk if it affects patient access to care).

Watch areas include patient flow systems that affect care delivery timing, resource allocation that determines who receives treatment, and triage systems that affect clinical prioritization.

When AI is a medical device

AI qualifies as a medical device when it is intended for diagnosis, prevention, monitoring, treatment, or alleviation of disease, provides information used for medical purposes, and falls within the MDR definition of medical device.

MDR Classification: Class I (lowest risk, self-certification), Class IIa (lower moderate risk, Notified Body required), Class IIb (higher moderate risk), Class III (highest risk). Most diagnostic and clinical AI falls into Class IIa or higher.

Coordinated conformity assessment

Article 6(1) of the EU AI Act provides that high-risk AI systems that are medical devices can satisfy EU AI Act requirements through MDR conformity assessment that incorporates EU AI Act requirements, integrated quality management system covering both regulations, and single Notified Body assessment addressing both frameworks.

Gap analysis: MDR vs. EU AI Act

Key gaps to address beyond MDR requirements:

• Human oversight documentation: MDR covers clinical validation; EU AI Act requires documented human oversight mechanisms
• Bias and fairness: MDR focuses on clinical safety; EU AI Act adds fairness and discrimination concerns
• Evidence integrity: EU AI Act requires verifiable evidence with integrity mechanisms
• Transparency to patients: EU AI Act has specific transparency requirements for AI systems

Clinician-in-the-loop requirements

EU AI Act requires that humans can understand the AI system's capabilities and limitations, monitor the AI system's operation, interpret the AI system's outputs correctly, decide to disregard, override, or reverse AI outputs, and intervene in the operation or stop the system.

Healthcare implementation includes clinical validation of AI outputs before action, clear display of AI confidence levels and limitations, override mechanisms for clinician judgment, and documentation of clinical review and decisions.

Override and escalation workflows

Design workflows for clinical AI that ensure meaningful human oversight with risk-based routing:

Clinical validation evidence

For EU AI Act compliance, maintain evidence of training and competency (clinician training on AI system use, understanding of limitations), operational oversight (review rates and patterns, override frequency and reasons, outcome tracking), and quality management (regular review of AI performance, incident tracking, continuous improvement).

Vendor assessment for AI tools

Before deploying third-party AI, assess EU AI Act compliance status (has the provider completed conformity assessment? Is CE marking in place?), human oversight support (does the system support clinical review workflows? Can clinicians override?), and evidence and audit requirements (what logging does the system provide? How is evidence retained?).

Contract requirements for AI vendors

Include in vendor contracts: compliance obligations (confirmation of EU AI Act conformity assessment, commitment to maintain compliance), evidence and documentation (access to technical documentation for audits, log retention, evidence export), and incident management (notification requirements, cooperation with regulatory inquiries, liability allocation).

Internal AI development governance

For health systems developing proprietary AI: development governance (AI ethics review process, clinical safety assessment, bias evaluation, documentation), deployment governance (clinical validation requirements, pilot and rollout procedures, monitoring), and lifecycle governance (change management, performance monitoring, incident response, decommissioning).

What healthcare regulators will ask

Expect questions on clinical safety (How do clinicians validate AI recommendations? What adverse events have occurred?), compliance (Which AI systems are medical devices? Have you completed conformity assessment?), governance (Who is accountable for AI governance? What training do clinicians receive?), and evidence (Can you show the AI role in specific patient decisions? How do you ensure evidence integrity?).

Multi-stakeholder evidence requirements

Build unified evidence collection that serves healthcare regulators (patient safety, clinical validation), EU AI Act authorities (conformity assessment, human oversight), MDR authorities (medical device compliance, post-market surveillance), accreditation bodies (quality management, clinical governance), and internal audit (control effectiveness, risk management).

Evidence retention and integrity

Healthcare-specific considerations: retention requirements (patient records typically 10+ years, medical device records 10+ years after last device on market, aligned with EU AI Act requirements), evidence integrity (tamper-evident storage for AI decision records, integration with clinical documentation systems, cryptographic verification, chain of custody documentation).