EDRi submits response to the European Commission AI adoption consultation

Today, 3rd of August 2021, European Digital Rights (EDRi) submitted its response to the European Commission’s adoption consultation on the Artificial Intelligence Act (AIA).

By EDRi · August 3, 2021

EDRi welcomes the European Commission’s globally significant step towards regulating the development and deployment of artificial intelligence systems. However, we would like to make suggestions to ensure that the AI Act is in line with the Charter of Fundamental Rights of the European Union (EU), future proof, and a role model for rights-protective future AI legislation around the world.

We raise concerns about the extent to which the Act protects fundamental rights and addresses broader structural, political and economic issues. In the attached paper, we highlight how the use of AI systems can systematically target, harm and exclude marginalised communities. We question how far the act can address these structural harms, due to its tendency towards de-regulation of all but the most narrowly-defined ‘unacceptable’ uses; the lack of obligations on users; and the lack of individual and collective redress for those subject to AI systems. EDRi thus recommends the following improvements:

1. Ensure effective protection against prohibited practices and address the full scope of unacceptable risks:
a) Strengthen art. 5 prohibitions to provide meaningful protection against fundamental rights violations and individual and collective harms;
b) Comprehensively prohibit biometric mass surveillance practices (RBI and automated recognition of human features in publicly-accessible spaces for any purpose);
c) Prohibit additional practices which are incompatible with fundamental rights and democracy: uses of AI in law enforcement or criminal justice that purport to predict future behaviour; in migration control in ways that undermine the right to claim asylum; to implement invasive surveillance, monitoring and algorithmic management in an employment and educational context; to categorise people on the basis of their human features; for emotion recognition; and in uses that constitute mass surveillance.

2. Adapt the AIA to ensure a holistic, democratic and future-proof framework:
a) Introduce a democratic, inclusive and accessible process for the insertion of new prohibitions, including criteria for ‘unacceptable risk’ and the addition of future prohibitions;
b) Ensure high risk ‘areas’ can be updated under art. 7;
c) Fix gaps relating to economic and environmental impact, structural inequality, migration control, law enforcement (LE), worker surveillance, mass surveillance and exports;
d) Remove loopholes in arts. 2.4 and 83 (international LE agreements);
e) Remove art. 47 broad public security exemption for conformity assessment;
f) Remove art. 54.1.a purpose limitation exemption in criminal justice.

3. Enhance obligations on users of all AI systems:
a) Mandate ex ante human rights impact assessments for high risk systems before putting into use;
b) Implement duty to cooperate with national authorities under arts. 65 and 67 for all systems;
c) Implement meaningful consultation duty;
d) Implement public authority notification requirement.

4. Implement meaningful public transparency for high risk systems:
a) Registration in EU database (art. 60) for AI systems that are put into use;
b) Add instructions for use for LE, migration, asylum and borders to public database;
c) Require providers to include access to conformity assessment as per article 13.2-3 under art. 60;
d) Require more thorough detail from providers to users under art. 13.3;
e) Remove art. 52 exemptions for transparency in detection, prevention or prosecution; for art. 52 uses, suspects should be notified post factum.

5. Facilitate accountability: Include oversight and enforcement infrastructures that work for people:
a) Ensure cohesive national enforcement structure;
b) Include individual and collective flagging and redress mechanisms;
c) Implement a more democratic governance infrastructure and greater independence for EAIB.

See EDRi’s paper for further analysis and recommendations here.

Sign the European Citizens’ Initiative (ECI)

If you're an EU citizen, you can help us change EU laws by signing the official #ReclaimYourFace initiative to ban biometric mass surveillance practices:

This is not a regular petition, but an official “European Citizens’ Initiative” (ECI) run by EDRi on behalf of the European Commission. This means your signature must be officially verified by national authorities, according to each EU country’s specific rules. We cannot control the data that they require, since it is required by Regulation (EU) 2019/788 on the European citizens’ initiative for the purpose of confirming your signature. We can only use the information that you provide in Step 2 to contact you with updates, if you choose to enter it. Furthermore, our ECI signature collection system has been verified by the German Federal Information Security Office (BSI) to ensure it is compliant with the EU’s Regulation on ECIs. Please see our “Why ECI?” page for further details, and check out our privacy policy.

This ECI is open to all EU citizens, even if you currently live outside the EU (although there are special rules for Germany). Unfortunately if you are not an EU national, the EU’s official rules say that you cannot sign. Check https://reclaimyourface.eu other ways than non-EU citizens can help the cause.

Note to German citizens: It is possible to sign our ECI petition if you live outside the EU, but German rules mean that for German citizens specifically, your signature will only be valid if you are registered with your current permanent residence at the relevant German diplomatic representation. If you are not registered, then unfortunately your signature will not be counted. You can read more information about the rules. This rule does not apply to citizens of any other EU country.

Legally, if we reach 1 million signatures (with minimum thresholds met in at least 7 EU countries) then the European Commission must meet with us to discuss our proposal for a new law. They must then issue a formal communication (a piece of EU soft law) explaining why they are or are not acting on our proposal, and they may also ask the European Parliament to open a debate on the topic. For these reasons, a European Citizens’ Initiative (ECI) is a powerful tool for getting our topic onto the EU agenda and showing wide public support for banning biometric mass surveillance practices.

Learn more about the campaign to ban biometric mass surveillance practices at our official website

Reclaim Your Face