EU AI Act needs clear safeguards for AI systems for military and national security purposes

AI-enabled national security and law enforcement measures and their impact on civic space

Recent years have seen a proliferation of AI in the field of national security and law enforcement. This trend has been reinforcing adverse effects on our civic freedoms and democratic processes. It has  also been negatively impacting the critical work of civil society, online and offline. The use of “security” technology has been known to target protestors (e.g., via biometric recognition) or limit activism (e.g., through removal of “terrorist” content on the internet). There is also a continuous demand for adapting the technology from the security and military arena to other fields (e.g., use of surveillance tools to detect terrorism is repurposed to monitor activism on the streets or adherence to pandemic rules). All this warrants a close watch and clear safeguards for civic freedoms while using AI for security related purposes.

Exemptions in the proposed AIA

However, the proposed AIA excludes AI systems developed or used exclusively for military purposes from its scope of application. Furthermore, the compromise text on the AIA approved by the Slovenian Presidency of the Council of the EU calls for an additional exemption regarding AI systems developed or used exclusively for national security purposes.

An (in) famous example of technology originally developed for national security only

The notorious case of NSO Group’s Pegasus spyware is the perfect example of a technology originally developed exclusively for national security purposes. In practice, however, this technology has also been used for law enforcement purposes – i.e., to help government agencies detect and prevent terrorism and crime – and even abused in those circumstances for spying ministers, journalists and dissidents/protesters among others. This resulted in civic space restrictions and human rights violations on a massive scale around the world.

What would be the consequences of excluding AI for “national security purposes” from the AIA?

Without proper regulatory safeguards, intrusive AI-based technologies – including with mass surveillance outcomes – could be used in the public sector with no special limitations or safeguards whenever “national security” grounds are invoked by a Member State. Even those AI systems presenting “unacceptable” levels of risks and therefore prohibited by the AIA could be easily “resuscitated” or “recycled” for the exclusive purpose of national security, affecting our freedoms of movement, assembly, expression, participation and privacy, among others.

In the context of developing technology, there is no clear line between law enforcement and national security. Therefore, AI developers will not be able to distinguish between an AI-based system to be used in an investigation related to an organised drug crime or a bomb threat.

ECNL’s legal briefer proposing amendments to Article 2 of the AIA (Scope)