Risk Management System

Article 9 is the cornerstone of high-risk AI compliance. It requires every provider of a high-risk AI system to establish, implement, document, and maintain a risk management system that operates continuously throughout the system's entire lifecycle -- from design through decommissioning.

The Four Steps of Article 9

Article 9(2) sets out the process: first, identify and analyse all known and reasonably foreseeable risks associated with the AI system. Second, estimate and evaluate the risks that may emerge from intended use, reasonably foreseeable misuse, and interactions with other systems. Third, adopt appropriate risk management measures to address identified risks. Fourth, test and evaluate the effectiveness of those measures and update them where necessary. The process must be documented and reviewed at least when the system undergoes a substantial modification.

Residual Risk and Testing

Article 9(6) requires that after implementing risk management measures, residual risk must be acceptable. Article 9(9) requires that high-risk AI systems be tested prior to being placed on the market or put into service, and that testing be performed against metrics and probabilistic thresholds appropriate to the intended purpose. Testing must address errors, faults, inconsistencies, and unexpected outcomes.

• Risk management must be continuous -- not a one-time pre-launch activity
• Reasonably foreseeable misuse must be considered, not just intended use
• Testing against probabilistic thresholds is required before market placement
• The risk management system feeds into technical documentation under Article 11
• Article 9(4) specifically requires consideration of risks to vulnerable groups including children