Table 6.

Overview of regulatory requirements pertaining to high-risk AI systems

Risk assessment and mitigation: developers of deceptive AI systems must maintain and regularly update a risk management system that identifies and analyzes relevant risks of ordinary use and misuse. These risks should be disclosed to users. Deceptive AI systems should be regularly tested for the extent of deceptive behavior during both development and deployment.

Documentation: developers must prepare technical documentation of the relevant AI systems and share with government regulators prior to the deployment of deceptive AI systems.

Record keeping: deceptive AI systems must be equipped with logs that automatically record the outputs of the system and must actively monitor for deceptive behavior. Incidents should be flagged to regulators, and preventive measures should be taken to prevent future deception.

Transparency: AI systems capable of deception should be designed with transparency in mind, so that potentially deceptive outputs are flagged to the user. Here, essential tools include technical research on deception detection, as well as bot-or-not laws.

Human oversight: deceptive AI systems should be designed to allow effective human oversight during deployment. This is especially important for future deceptive AI systems incorporated into management decisions.

Robustness: AI systems with the capacity for deceptive behavior should be designed with robust and resilient backup systems, ensuring that, when the system behaves deceptively, backup systems can monitor and correct the behavior. It is also crucial to insulate deceptive AI systems from critical infrastructure.

Information security: adversaries may be interested in stealing models with deceptive capabilities. Developers should be required to implement rigorous information-security practices to prevent model theft.

The regulatory requirements are listed in Title III of the EU AI Act.⁸¹