EU KI-Verordnung Referenz
Systemkonfiguration
Regulation (EU) 2024/1689 · Articles 9–15, 17, 43, 72
114 / 114 controls
Article 9(1) requires providers to establish, implement, document, and maintain a risk management system as a continuous iterative process throughout the entire lifecycle of the high-risk AI system.
Providers must identify and analyse known and foreseeable risks that the high-risk AI system may pose to health, safety, or fundamental rights when used as intended and under reasonably foreseeable misuse conditions.
Providers must estimate and evaluate the likelihood and severity of potential harm, taking into account the intended purpose, foreseeable misuse, and the vulnerability of affected natural persons.
Providers must evaluate identified risks against pre-defined risk acceptance criteria and document the rationale for accepting residual risks that cannot be fully mitigated.
Providers must adopt suitable risk management measures to address identified risks, prioritising the elimination or reduction of risk at design stage before implementing protective measures.
Article 9(7) requires that high-risk AI systems are tested to identify the most appropriate risk management measures and to verify that the system performs consistently with its intended purpose throughout development.
Providers must document residual risks that users need to be informed of and include relevant information in the system instructions for use, enabling deployers to manage remaining risks appropriately.
The risk management system must continuously monitor the AI system in operation to identify new or evolving risks and trigger appropriate reassessment when substantial modifications occur or when post-market data indicates new hazards.
Providers must conduct periodic reviews of the risk management system and documentation to ensure it remains appropriate, adequate, and effective throughout the system lifecycle and following any significant change.
Providers must communicate relevant risk information to deployers, affected persons, and competent authorities as required, including risk summaries in the instructions for use and technical documentation.
Article 9(4) requires that risk management measures applied to high-risk AI systems address both known and foreseeable misuse scenarios, with particular attention to unintended use by persons who lack relevant technical expertise.
Providers must assess risks arising from the training data used to develop the AI system, including risks of bias, data quality defects, and gaps in representativeness that could affect the system's performance on affected persons.
Providers must assign clear roles and responsibilities for risk management activities, ensuring that personnel performing risk management tasks have the necessary competency, resources, and authority to fulfil their obligations.
The risk management process must be integrated into the full AI system lifecycle, from design and development through deployment and decommissioning, with documented handover points and responsibilities at each phase transition.
Article 10(3) requires that training, validation, and testing data sets are subject to data governance practices that ensure relevance, representativeness, and freedom from errors to the extent possible given the intended purpose.
Providers must examine training, validation, and testing datasets for possible biases that could affect health, safety, or fundamental rights, and implement appropriate bias mitigation measures before and during model training.
Providers must document the origin, collection methodology, labelling process, and relevant characteristics of all data sets used in training, validation, and testing to enable traceability and reproducibility.
Article 10(3) requires that training data sets are sufficiently representative of the intended population and use-case context, and providers must document the demographic and situational coverage of training data.
Providers must verify that data sets used for high-risk AI systems are relevant and complete for the system's intended purpose, documenting any known gaps and their potential impact on system performance.
Where data labelling or annotation is performed, providers must implement quality assurance procedures to ensure consistency, accuracy, and reproducibility of annotations, including inter-annotator agreement measurement.
When processing personal data for AI training, providers must implement appropriate privacy-preserving techniques such as pseudonymisation, anonymisation, or differential privacy, in compliance with GDPR Article 25.
Providers must maintain versioned records of training, validation, and testing datasets used in each version of the AI system, enabling reproducibility of results and traceability of data changes that may affect system behaviour.
Providers must apply appropriate statistical methods to validate that training data meets quality thresholds and that the resulting model generalises appropriately to the intended deployment distribution.
Providers must establish and document data retention periods for training and operational data, with deletion procedures that comply with GDPR storage limitation principles and enable data subject rights fulfilment.
Where training or inference involves personal data, providers must maintain records of processing activities as required by GDPR Article 30, describing the categories of data, purposes of processing, and technical safeguards in place.
Article 10(5) permits processing of special categories of personal data for bias monitoring and correction only under strict conditions; providers must document the legal basis, safeguards, and scope limitations for any such processing.
Article 11 and Annex IV require that technical documentation includes a general description of the AI system covering its intended purpose, the persons responsible for it, and the characteristics, capabilities, and limitations of the system.
Providers must document the overall design logic of the AI system, including the algorithms and associated design choices, the key design parameters, and the interaction between the AI components and other system elements.
Technical documentation must describe the methods and steps performed for the development of the AI system, including data collection, labelling, model training, and evaluation procedures with their rationale.
Providers must document the validation and testing procedures applied to the AI system prior to placing it on the market, including the metrics used, the test sets employed, and the results achieved.
Technical documentation must include a plan describing the measures for monitoring the AI system in operation, the data to be collected, the performance thresholds, and the procedures for responding to performance deviations.
Providers must document the accuracy, robustness, and cybersecurity metrics for which the high-risk AI system was designed and the relevant testing methodologies, including the context in which these metrics were measured.
Article 11 and Annex IV require explicit documentation of known or foreseeable limitations of the AI system, including performance degradation conditions, out-of-distribution failure modes, and contexts where the system should not be used.
Technical documentation must be kept up to date with each version of the AI system and include a version history documenting all substantial modifications, the rationale for changes, and their impact on system performance and compliance.
Providers must document procedures for managing changes to the AI system that could affect its compliance status, triggering re-assessment when a modification is substantial and potentially requiring new conformity assessment.
Providers must establish a formal review and approval process for technical documentation to ensure accuracy, completeness, and continued applicability, with records of each review cycle maintained for at least 10 years.
Article 12(1) requires that high-risk AI systems are designed and developed with automatic logging capabilities, enabling the reconstruction of events relevant to identifying risks and substantial modifications throughout the system's lifetime.
Providers must ensure that logs are protected against tampering, unauthorised deletion, or modification, using cryptographic integrity controls, write-once storage, or equivalent technical measures.
Article 18(1) requires that providers retain technical documentation and logs for at least 10 years after the high-risk AI system is placed on the market or put into service, to enable post-market surveillance and authority access.
Providers must ensure that event logs are accessible to competent national authorities and market surveillance authorities upon request, with procedures for secure and timely disclosure of relevant log records.
Logs must enable the tracing and reconstruction of decision events to understand the inputs processed, the outputs generated, and the conditions under which the AI system operated at any given point in time.
Providers must log performance and operational metrics relevant to verifying that the AI system operates within the parameters established in the technical documentation, including accuracy and response time indicators.
The logging system must capture anomalous behaviour, errors, and unexpected outputs from the AI system in operation, enabling timely detection of performance degradation and triggering corrective action procedures.
Where human oversight decisions are made in connection with the AI system's outputs, providers must log these decisions and their rationale to enable review of the effectiveness of human oversight measures.
Providers and deployers must log all incidents, near-misses, and serious incidents involving the AI system, with sufficient detail to support root cause analysis and to meet the reporting obligations under Article 73.
Providers must maintain a complete audit trail for each conformity assessment, including all documentation reviewed, tests performed, results obtained, and decisions made, to support regulatory scrutiny and re-assessment activities.
Article 13(1) requires providers to supply high-risk AI systems with instructions for use containing the information necessary for deployers to understand the system's capabilities, limitations, and conditions for safe and effective deployment.
Providers must clearly communicate the intended purpose of the AI system, the specific context and use cases for which it was designed, and the categories of natural persons and other entities that may be affected by its use.
The instructions for use must include the performance metrics for which the system has been designed and the level of accuracy that the provider committed to achieving, enabling deployers to assess whether the system meets their operational requirements.
Providers must include in the instructions for use any known or foreseeable circumstances in which the AI system may fail to perform as intended, conditions of use that may lead to risks, and the populations or contexts for which the system was not designed.
Instructions for use must provide deployers with information about human oversight measures required, including the competencies needed to interpret outputs, the circumstances requiring human intervention, and how to use the override capability.
The instructions for use must describe the form and type of data required as inputs to the AI system, the formats expected, and any constraints on input quality or content that could affect system performance.
Providers must ensure AI systems intended to interact directly with natural persons are designed and developed so those persons are informed that they are interacting with an AI system, unless this is obvious from the context or a paragraph-specific exception applies.
Article 50(2) requires providers of AI systems generating synthetic audio, image, video, or text content to mark outputs in machine-readable format and make them detectable as artificially generated or manipulated.
Operators deploying emotion recognition systems must inform natural persons exposed to such systems that they are subject to an emotion recognition system, with appropriate notice provided in advance.
Deployers of AI systems that generate or manipulate image, audio, or video content constituting a deepfake must disclose that the content has been artificially created or manipulated, with the disclosure adapted to the media context.
Providers must implement measures that enable deployers and, where feasible, affected persons to understand the basis for AI system outputs to the extent technically possible, supporting informed human oversight decisions.
Providers must include in the instructions for use the name and registered address of the provider and, where applicable, their authorised representative in the EU, along with contact details for support and regulatory inquiries.
Article 14(1) requires that high-risk AI systems are designed and developed in a way that enables natural persons to effectively oversee the system's operation, with oversight tools built into the system design rather than added as an afterthought.
High-risk AI systems must include a capability for human overseers to intervene in or interrupt the system's operation, and providers must document this capability in the instructions for use with procedures for its activation.
Providers must implement a stop function that enables the immediate cessation of the AI system's operation when required by human overseers, with a clear, accessible interface that does not require technical expertise to activate.
Article 14(4) requires that oversight persons are aware of the tendency to over-rely on AI outputs (automation bias); deployers must implement training and procedural controls to counteract bias toward uncritical acceptance of AI decisions.
Deployers must ensure that persons assigned to oversee high-risk AI systems have the capability to correctly interpret the system's outputs, including the ability to recognise when outputs may be unreliable or outside the intended operating range.
Deployers must establish documented procedures for reversing or correcting AI-assisted decisions that affect natural persons, including how affected persons can request human review and how such requests are processed.
Providers must specify in the instructions for use the competencies required for persons assigned to oversee the AI system, and deployers must verify that assigned personnel meet these requirements before deployment.
Deployers must provide training to persons responsible for overseeing AI systems, covering the system's intended purpose, limitations, risk factors, and the specific oversight procedures required, with records of training completion.
Deployers must define and communicate escalation procedures for situations where the AI system behaves unexpectedly, produces outputs that raise concerns, or where human overseers require additional expertise or authority to act.
Providers and deployers must periodically review the effectiveness of human oversight measures, including assessing whether the designated oversight persons are able to meaningfully intervene and whether oversight tools remain fit for purpose.
Article 15(1) requires that high-risk AI systems are designed and developed to achieve an appropriate level of accuracy, robustness, and cybersecurity, with providers defining target accuracy levels in technical documentation.
Providers must measure and verify the accuracy of the AI system against defined metrics using representative test sets, documenting the methodology, conditions, and results achieved in technical documentation.
Providers must test the AI system's robustness against input variations that may arise in real-world deployment, including incomplete data, noisy inputs, and distribution shifts from the training environment.
Where relevant to the AI system's use case, providers must test robustness against adversarial inputs designed to cause misclassification, data poisoning during training, or other security-related failure modes.
Article 15(5) requires that high-risk AI systems are resilient against unauthorised third-party attempts to alter their outputs, and providers must implement appropriate cybersecurity measures throughout the system lifecycle.
Providers must design and implement fallback procedures that activate when the AI system cannot operate within its designed parameters, ensuring safe degradation of service and notification to human overseers.
Providers must implement input validation mechanisms that verify the quality, format, and plausibility of data before it is processed by the AI system, rejecting or flagging inputs that fall outside acceptable parameters.
Providers and deployers must monitor AI system performance in operation for signs of degradation below acceptable thresholds, with alerts configured to trigger human review and corrective action procedures.
For AI systems that continue learning after deployment, providers must implement safeguards that prevent performance degradation through feedback loops, ensuring that any self-learning is subject to validation before influencing outputs.
Providers must conduct stress testing of the AI system under conditions of high load, unusual inputs, and edge cases, documenting results and ensuring the system behaves safely and predictably at the limits of its operating parameters.
Article 17(1) requires providers of high-risk AI systems to put in place a quality management system that ensures compliance with the requirements of the EU AI Act, documented in policies, procedures, and instructions.
Providers must establish and document a strategy for achieving and maintaining compliance with applicable regulatory requirements, including the EU AI Act, GDPR, and applicable national sector laws, with defined responsibilities and timelines.
Providers must implement documented design control procedures that ensure regulatory and performance requirements are systematically incorporated from the earliest design stages and verified at each major development milestone.
Providers must have documented data management procedures covering the acquisition, preparation, use, and retention of data throughout the AI system lifecycle, aligned with Article 10 data governance requirements.
Article 17(1)(d) requires providers to implement procedures for training personnel involved in AI system development, testing, and monitoring, with records of training completions and periodic competency reassessment.
Providers must implement documented procedures for pre-market testing and validation of high-risk AI systems, including the metrics to be achieved, the testing environments, and the acceptance criteria that must be met before market release.
Article 17(1)(f) requires providers to implement a post-market monitoring plan that specifies the data to be collected, the monitoring frequency, the performance thresholds, and the feedback mechanisms for incorporating operational insights.
Providers must identify, allocate, and document the resources necessary for operating the quality management system, including personnel, infrastructure, technology, and financial resources required for sustained compliance.
Providers must establish a clear accountability framework identifying senior management responsibility for AI compliance, with defined governance structures that ensure escalation of compliance issues to appropriate decision-making levels.
Providers must implement procedures for continuous improvement of the quality management system, including internal audits, management reviews, nonconformity management, and corrective action processes.
Providers must implement document control procedures that ensure QMS documents are current, approved, accessible to relevant personnel, and subject to version control, with obsolete documents retired from active use.
Senior management must conduct periodic reviews of the quality management system to assess its continuing suitability, adequacy, and effectiveness, with documented outputs including decisions on improvement actions and resource needs.
Article 43 requires providers to select the appropriate conformity assessment procedure for their high-risk AI system: internal control (Annex VI) under Article 43(2) for Annex III points 2 to 8, or third-party assessment by a notified body (Annex VII) under Article 43(1) for Annex III point 1 biometric systems where harmonised standards were not fully applied.
For AI systems subject to internal control assessment, providers must verify and document that the system meets all applicable requirements of Articles 9–15 and 17, with the internal assessment conducted by qualified personnel independent of the development team.
Third-party assessment is the exception. It applies to Annex III point 1 biometric identification and categorisation systems under Article 43(1) where harmonised standards were not fully applied, and to Annex I Section A products under Article 43(3), where the notified body already designated under the sectoral legislation assesses the AI Act requirements. Article 43(2) is the internal-control route and involves no notified body. Employment, credit, education, and essential-services systems fall under Article 43(2).
Article 48 requires that providers affix the CE marking to high-risk AI systems that have successfully completed conformity assessment, certifying that the system meets applicable EU AI Act requirements and is fit for placing on the EU market.
Article 47 requires providers to draw up an EU Declaration of Conformity for each high-risk AI system, declaring that it meets all applicable requirements, and to keep it updated for 10 years after the system is placed on the market.
Where third-party assessment is required, providers must select a notified body that is duly designated for the relevant Annex III category, verify its accreditation status in the Commission NANDO database, and formalise the engagement before assessment commences. Notification is Union-wide under Articles 30 and 35, so a body designated by any Member State may be used.
Article 43(4) requires providers to re-assess conformity when a substantial modification is made to the AI system, including changes that affect the intended purpose, the system's performance characteristics, or its classification under Annex III.
Providers must prepare a complete technical file comprising all technical documentation required under Article 11 and Annex IV, organised to support efficient review by market surveillance authorities or notified bodies.
Article 21 requires providers and deployers to cooperate with national market surveillance authorities upon request, providing access to technical documentation, logs, and the AI system itself as needed for supervisory activities.
Providers must establish procedures for identifying non-conformities during conformity assessment or post-market surveillance, implementing corrective actions, and re-assessing compliance before re-releasing any modified system.
Providers placing high-risk AI systems on markets in multiple EU Member States must ensure that the conformity assessment is valid across all jurisdictions, liaising with the AI Office and relevant national authorities as necessary.
Article 49 requires providers to register standalone high-risk AI systems listed in Annex III in the public EU AI Act database before placing them on the market, providing specified information through the registration portal.
Article 72(3) requires providers to establish, document, and implement a post-market monitoring plan that proportionately reflects the nature of the AI technology and the risks posed by the high-risk AI system in its intended purpose.
Providers must implement a system for collecting and analysing relevant data on the performance of high-risk AI systems throughout their operational lifetime, including data provided by deployers under their cooperation obligations.
Providers must continuously track performance indicators in operation, comparing actual performance against the baseline established during pre-market validation, and documenting trends that could indicate emerging compliance concerns.
Providers must implement automated and manual mechanisms to detect incidents and near-misses in operation, and conduct root cause analysis for significant incidents to identify systemic issues and prevent recurrence.
Article 73(1) requires providers and deployers to report serious incidents to the relevant market surveillance authority without undue delay after becoming aware of them, with initial reports submitted within the timeframes specified in Article 73(2).
Providers must implement documented corrective action procedures that specify how identified performance issues, incidents, and non-conformities are addressed, including investigation, action determination, implementation, and effectiveness verification.
Providers must establish channels for receiving and systematically processing feedback from deployers about AI system performance, incidents, and concerns, integrating relevant insights into the risk management and post-market monitoring systems.
Providers must have documented procedures for recalling or withdrawing high-risk AI systems from the market when a serious risk to health, safety, or fundamental rights is identified that cannot be adequately mitigated through corrective actions.
Providers and deployers must establish protocols for cooperating with national market surveillance authorities during post-market surveillance activities, including data sharing, on-site access arrangements, and response time commitments.
Providers must conduct periodic reviews of post-market monitoring data and the effectiveness of the monitoring system, producing documented review reports that inform updates to risk management, technical documentation, and the monitoring plan itself.
When serious risks are identified through post-market monitoring, providers must notify affected deployers and, where required by competent authorities, implement measures to inform affected natural persons of the risk and available remedies.
Providers must periodically validate that the post-market monitoring system is operating correctly and capturing the data necessary to detect performance issues in a timely manner, including end-to-end testing of the data collection and alerting pipelines.