While offering vast opportunities for exercising and enhancing fundamental rights, the digital environment also offers both opportunities to commit new offences and to impose new restrictions on our online rights. Measures such as filtering, blocking and untargeted surveillance are often easy to implement and extremely difficult to rectify. EDRi therefore works to ensure that all security and surveillance measures are necessary, proportionate and implemented based on solid evidence.
European Digital Rights is proud to announce that Sarah Chander has joined the team at the Brussels office as the new Senior Policy Advisor. Sarah will work as Senior Policy Adviser for EDRi, focusing on AI, discrimination, Adtech and hate speech, amongst other issues. In addition, she will explore intersections between EDRi’s work and other social issues.
Sarah has experience in in social movements and civil society efforts and is excited to enter the digital rights field. Previously she worked in advocacy at the European Network Against Racism (ENAR), on a wide range of topics including anti-discrimination law and policy, hate crime and speech, racial profiling, and diversity and inclusion. Before that she worked on youth employment policy for the UK civil service. She was actively involved in movements against immigration detention. She holds a masters in Migration, Mobility and Development from SOAS, University of London and a Law Degree from the University of Warwick.
At ENAR Sarah began to explore how the anti-racism and digital rights field intersect, in particular how tech is increasingly used by police and immigration control across Europe, and also through AI in the field of recruitment. You can read more about this in her blog post on data racism, found here in English and here translated in German.
The European Parliament Committee on Civil Liberties (LIBE) is currently busy working out a compromise between its different political groups in order to establish a common position on the “e-evidence” Regulation. It is an important step of the legislative process since the Parliament’s position will be the only bulwark standing between the proper protection of human rights in cross-border law enforcement and the Council’s and the Commission’s highly problematic e-evidence ideas.
To ensure that fundamental rights are protected when law enforcement authorities in an EU Member State act outside their own country, the Parliament compromise should do the following:
✅ Do: The authority in the executing Member State – and where applicable the affected State – should be obliged to confirm or reject an order before the online service provider can execute it. Some compromises on the table suggest that after a certain period of time (for now, 10 days) without a reaction from the executing authority, the service provider should simply assume green light. This is a risky shortcut because it creates the incentive for an underfunded and understaffed executing authority to ignore requests and let the 10-day deadline pass without action. Given that many service providers are currently based in Ireland, whose law enforcement authorities are disproportionately flooded by these requests, this is a very real scenario which would practically annul many of the otherwise very important human rights safeguards built in by the Member of the European Parliament (MEP) Birgit Sippel, Rapporteur of this file in LIBE Committee. In practice, it would mean that the service providers become the ultimate safety net against abusive requests and fundamental rights breaches. The explicit approval by the executing authority must therefore be mandatory before an order can be executed.
❌ Don’t: Some compromise proposals imply that orders to access subscriber data would have no suspensive effect on the handing out of data. Those proposals assume that – if an order is found invalid at a later state – the data could simply be declared inadmissible in court and be deleted by the issuing authority. In practice, however, this notion is misleading, at best. Once an investigating police officer learns a suspect’s identity, they will not be able to unknow that identity just because the subscriber data they learned it from has been declared inadmissible and is deleted. What is worse, if indeed the knowledge of the identity itself was declared inadmissible, the whole investigation would collapse. In order to ensure legal certainty for investigating officers, again, an order should not be executed without the authorisation of the executing authority. This procedural requirement should apply for all types of data and orders.
✅ Do: Just like the service provider is given the possibility to refuse an order because it is manifestly abusive, the executing authority should be obliged to check that the order is proportionate as part of its refusal grounds checklist.
❌ Don’t: Considering the state of the rule of law in certain EU Member States, executing an order from a State where the independence of the judiciary is not guaranteed would be incredibly risky for the protection of fundamental rights. In line with the jurisprudence of the Court of Justice of the EU on the independence of judicial authorities, any executing authority should refuse to execute orders from States subjected to Article 7 proceedings.
If adopted, these changes would strengthen the Parliament’s Report and ensure it is able to defend citizens’ rights during the e-evidence negotiations with the Commission and the Council.
From a fundamental rights perspective, it’s essential that the proposal enabling cross-border access to data for criminal proceedings (“e-evidence”) includes a notification mechanism. However, this requirement of a notification seems to be out of the question for those advocating for “efficiency” of cross-border criminal investigations, even if that means abandoning the most basic procedural safeguards that exist in the European judicial cooperation. Another argument against notifying at least one second authority is that the system would be “too complicated”.
To solve these intricacies, others (with similar goals in mind) have proposed to restrict the scope of the future legal instrument to “domestic” cases only. This means that the Production and Preservation Orders would only be used in cases where the person whose data is being sought is residing in the country of the issuing authority. For example, if a French prosecutor is investigating a case involving a French resident as a suspect, she would not need to request the approval of another State. Only the executing State – where the online service provider is established – would be required to intervene exclusively in cases where the provider refuses to execute the Order.
“Domestic” cross-border cases?
Traditionally, EU Member States execute their own national laws to summon service providers established on their territory to hand over their customers’ data for criminal matters. However, when the evidence that is being searched for is located in another Member State, European Union’s rules for judicial cooperation kick in. The new regime, as proposed by the EU Commission, allows Member States to extend their judicial jurisdiction beyond their territorial boundaries. This affects the sovereignty of other states – and this is why we talk about cross-border access to data and cannot refer to “purely domestic cases”.
The notification of a second authority, notably the executing authority, is essential for the new instrument to be compatible with EU primary (the EU treaties) and secondary law. For the principle of mutual recognition – which is the legal basis chosen for the proposal – to apply, it is indeed crucial that the executing State is first aware that an order is executed on its territory, before it is able to recognise it.
Notification to the executing authority
Some stakeholders in the e-evidence debate grumble about the administrative burden that a notification procedure would entail. They further underline the problematic situation it would create for Irish judicial authorities since Ireland hosts so many prevailing service providers. However, there are several counter-arguments to these claims:
The proposal does not solely cover circumstances in which Ireland will be involved but all cross-border cases in the EU;
It is vital to design policies that are future-proof. The current lack of European harmonisation in the field of taxation policies impacts the efficiency of judicial cooperation instruments. This doesn’t, however, mean it will always be that way. It is also not a satisfactory justification for bypassing fundamental rights safeguards;
Service providers should not be required to execute orders that are unlawful under the law of the country where they are located;
In cases where the affected person’s residence is unknown, a notification mechanism to access identifying information would be imperative, because there would be no indication at this stage of the investigation as to whether there is a third State concerned or not.
There are cases in which the affected person is residing somewhere else than in the issuing State. Excluding these cases from the e-evidence proposal’s scope would allow a relief for law enforcement authorities from a couple of additional legal checks – that would allow them, in particular, to avoid the verification of immunities or other specific protections that are granted by national laws, and that restrict the access to certain categories of personal data. Nonetheless, excluding those cases would not suffice to meet the obligation to respect fundamental rights and rule of law standards provided in the EU legal system when using mutual recognition instruments. Instead, a notification mechanism with an obligation to confirm or refuse the order for both the executing and the affected States should feature in the final text.
On 18 February 2020, the Swedish parliament passed a law that enables Swedish law enforcement to hack into devices such as mobile phones and computers that the police thinks a suspect might use. As with the recent new data retention law only one party (and one member of another party) voted against the resolution (286-26 with 37 absent). The previous data retention law was struck down, and given the directions of the recent Court of Justice of the European Union (CJEU) Advocate General (AG) Opinions on data retention, the current data retention law is likely to be struck down as well.
What capabilities does this give law enforcement agencies?
For crimes that “under the circumstances” can reasonably give at least a two-year prison sentence, law enforcement agencies (LEAs) can request a court warrant to hack into the suspect’s device. This warrant can be given to gather information (for example from encrypted messaging apps) or even in some cases to stop information from being sent from that device.
The law has a number of serious issues that has been pointed out to lawmakers over several years when the law was going through the public inquiry phase. For example, the law does not say that a minimum sentence of two years in prison is required, but that if the prosecutor just believes that the suspected crime might carry two, or more, years in prison, that already give LEAs the legal basis to ask for a court warrant.
Even more worryingly, even citizens who are not suspect of having committed any crimes, but are associated with a suspect might be the targets of hacking by the police. The law gives the LEAs a mandate to hack devices that they reasonably think a suspect might primarily use. So if a suspect might uses their mother’s phone, for example, that device is open to hacking. If you are someone that the police think their suspect will call or message, your phone might also be in danger of being hacked, just because you happen to know someone that the police suspects of a crime. They can also be allowed to use hacking to find a suspect – this means you simply shouldn’t be at the wrong place at the wrong time, or else the police might hack your devices.
The law also includes a clause that states that if the prosecutor feels like the courts will be too slow to issue a warrant, he or she can issue it. If the court then finds that the warrant was wrongly issued, the prosecutor will then have to go to court for review, and any evidence gathered can not be used against the suspect. Of course, the person whose device was hacked (who might not even be a person suspected of a crime) has already had their privacy breached, and the law doesn’t provide any recourse for such abuses.
The new law goes into effect on the 1 April 2020 and will be valid for five years, after which the Swedish parliament will decide to make it permanent or not.
On 18 February 2020, the Romanian Constitutional Court unanimously declared unconstitutional a new legislative act adopted in September 2019 introducing mandatory SIM card registration. The legislative act in question was an emergency ordinance issued by the Government which wanted to introduce this obligation as a measure “to improve the operation of the 112 emergency service number”. This is the second time the court issues an unconstitutionality decision on mandatory SIM card registration proposals.
The court dismissed the law on procedural grounds, as the government failed to demonstrate the emergency of adopting the ordinance. It also highlighted in its press release that this act of introducing the SIM card registration provision has actually been postponed twice, therefore the urgency to issue an emergency ordinance was non-existent.
Coincidentally or not, two days later, a false bomb threat (the first in years) was reported in the media. A man called the 112 emergency number claiming that he placed a bomb inside a shopping mall. The call was made from a prepaid SIM card, a fact that the 112 service specifically highlighted in their press release, while amping up about the number of staff involved in responding to this call, subtly suggesting the implications of this types of calls are massive, and if they are proven false, there is nobody that can be taken responsible as prepaid SIM card calls can’t be traced back to an individual.
After a tragic failure by the police to save a teenage girl who was abducted but managed to call the 112 emergency number three times before she was murdered, the Romanian Government adopted an Emergency Ordinance which introduced the obligation to register prepaid SIM cards.
EDRi member ApTI, together with the Association for the Defence of Human Rights in Romania – the Helsinki Committee (APADOR-CH), asked the Romanian Ombudsman to send the law to the Constitutional Court. The Ombudsman challenged the case in court and ApTI sent an amicus curiae in support the unconstitutionality claims, showing that:
there was no reason to justify the adoption of an emergency ordinance instead of going through the normal parliamentary procedure;
restricting individual freedoms by requesting to register prepaid SIM cards is a measure disproportionate to the goal that was intended – to limit fake calls to the 112 emergency number;
no data protection impact assessment has been carried out and the national data protection authority did not support the law;
the Constitutional Court already decided in 2014 that the measure to introduce mandatory SIM card registration limits the fundamental rights and freedoms and such measures can only be introduced if they are necessary and proportionate.
The second phase of the stakeholder dialogues on Article 17 of the Copyright Directive finished in December 2019. The two meetings of the third phase, focusing on the provisions of Article 17, were held on 16 January and 10 February 2020.
The concepts of authorisation and best efforts were discussed in the first meeting on 16 January. The second meeting on 10 February covered users’ safeguards and redress mechanisms.
Can we still fix Article 17?
On 10 February, for the first time, the dialogue gave the opportunity to discuss the central issue of Article 17 (former Article 13): the conflict between platforms’ obligations and users’ rights.
Article 17(4) requires platforms to make best efforts to avoid the availability of copyright protected content, which in practice forces the implementation of upload filters. On the other side, Article 17(7) ensures that the cooperation between rightsholders and platforms does not lead to the blocking or removal of legitimate content, such as copyright exceptions. It has been demonstrated, even during these dialogues, that filters cannot understand context and therefore cannot recognise when users make use of copyright exceptions such as criticisms or parody. Stakeholders discussed how this conflict could be addressed.
Users’ organisations presented concrete scenarios to avoid the automated blocking of content
How could this conflict be solved in practice? EDRi supports a scenario where the content flagged by filters would stay online during the whole process of review, from the moment when being uploaded, until human review, as seen in the image below.
Communia, an NGO promoting policies that expand the public domain, presented a slightly different scenario, based on recommendations from academics. They differentiate between two types of copyright infringement flagged by filters:
First, a “prima facie” copyright infringement, an identical or equivalent match to a protected content, which would lead to the content being automatically blocked. However, users would still be able to complain and be entitled to the safeguards of Article 17(9). Because of the potential for abuse, the proposal also includes possible sanctions for rightsholders who repeatedly make wrongful ownership claims.
Second, in cases of partial matches to protected content, users would be able to make declarations of lawful use, allowing their content to stay up during the whole process, until the human review.
The European Consumer Organisation BEUC also suggested that users should be educated about copyright to allow them to fully make use of this declaration.
Surprisingly, Studio 71, a global media company owned by broadcasters, presented a similar model based on the same academics paper. The difference with Communia’s scenario is the introduction of a “red button” that rightsholders would be able to use to immediately block content, even after a user makes a declaration of lawful use. Studio 71’s proposal also includes sanctions for both users and rightsholders in case of repeated abuse. They even provided evidence of their own overblocking, saying that they allegedly once wrongfully blocked 50 000 pieces of content.
The European Commission showed interest and kept asking for more details especially regarding the concrete implementation of a prima facie rule. However, it quickly became clear that most rightsholders had no interest in joining such discussion, and instead insisted that their rights should prevail and that the complaint and redress mechanism was the only solution to overblocking.
This is not the first time that stakeholders refuse to meaningfully engage in the dialogue process. On 16 January, most of them refused to provide useful input to help define important concepts of Article 17. Rightsholders have not shown interest in guidelines that would limit their blocking capabilities, and big platforms already manage their own blocking tools.
European Commission to the rescue?
Faced with the rightholders’ attitude, the European Commission explained that their role in this dialogue was to ensure the effectiveness of the exceptions. They believe that the interpretation of Article 17 where the only remedy available for users is the redress mechanism might not ensure such effectiveness. They quoted the Court of Justice’s case law, saying that exceptions are particularly important for freedom of expression. They then strongly encouraged the rightsholders to suggest workable concrete solutions, instead of shutting down any attempt at compromise.
However, despite the Commission’s intervention, we are still in the dark regarding the content of the guidelines and whether they will actually be able to safeguard users’ rights against powerful commercial interests. Moreover, will the guidelines be followed by EU Member States when they implement the Directive? Some Member States already drafted their implementation laws, and users’ rights safeguards are missing. The Commission must actively ensure that national implementations of the Directive effectively protect users’ rights.
The European Commission said it will reflect on the input received during these six meetings with stakeholders, but wants to meet up again around 30 March. This final meeting would be the opportunity to present their initial view on the content of the guidelines. They will also to launch a written and targeted stakeholder consultation, details on which will be provided at the meeting in March.
We hope that this means the whole draft guidelines will be shared with us and that the purpose of this consultation will be to seek feedback on whether the document can be further improved to ensure compliance with the Charter of Fundamental Rights of the EU, as we requested in January 2020.
On 13 February 2020, the European Court of Human Rights (ECtHR) issued its judgment in the case Gaughran v. The United Kingdom (UK), on the indefinite retention of personal data (DNA profile, fingerprints and photograph) of a man who had a spent conviction. The Court ruled that in the case of the applicant, the retention at issue constituted a disproportionate interference and therefore a violation of his right to respect for private life (Art. 8 of the European Convention on Human Rights) since the interference could not be regarded as necessary in a democratic society.
The Court relied much of its arguments on the S. and Marper v. The UK (2008) case where it found that the blanket and indiscriminate nature of the powers of retention of the fingerprints, cellular samples and DNA profiles of the applicants, suspected but not convicted of offences, as set out in the UK law, breached their right to respect of private life. Subsequently, the relevant legislation was amended in England and Wales, but not in Northern Ireland, where the applicant in the Gaughran case committed the criminal offence for which he was convicted.
Indefinite retention regime and lack of necessary and relevant safeguards
According to the ECtHR case law, the Court does not necessarily or solely focus on the duration of the retention period to assess the proportionality of a measure. The Court will rather look at whether the regime takes into account the seriousness of the offending and the need to retain the data, as well as the safeguards available to the individual.
In this case, because the UK chose to put in place a regime of indefinite retention, the Court argues that there is therefore “a need to for the State to ensure that certain safeguards were present and effective for the applicant” (para. 94).
The Court pointed out in this regard that the lack of any relevant safeguards. First, the biometric data and the photograph of the applicant were retained “without reference to the seriousness of his offence and without regard to any continuing need to retain that data indefinitely”. Moreover, the Court noted the absence of any real review available to the individual, as well as the police, which can only delete the data in exceptional circumstances (para. 94).
The UK overstepped the acceptable margin of appreciation
Part of the proportionality assessment is also looking at the margin of appreciation of the state. Similar to the national security argument, a wide margin of appreciation is often invoked by the governments to justify measures interfering with fundamental rights.
The Court found the margin to be way narrower than what the UK claimed, and still not wide enough to conclude that the retention was proportionate. Contrary to what the UK stated, the Court states (para. 82-84) that the majority of states have regimes in which there is a defined retention period. Furthermore, it states that the UK is one of the few of the Council of Europe jurisdictions to permit indefinite retention of DNA profiles, fingerprints and photographs of convicted persons.
Moreover, the UK claimed that indefinite retention measures were relevant and sufficient as the more data is retained, the more crime is prevented. This dangerous and false narrative was challenged by the Court as this would justify the “storage of information on the whole population and their deceased relatives, which would most definitely be excessive and irrelevant” (para. 89).
Beware of evolving technologies
One of the UK’s argument regarding the proportionality of the measure was that the applicant’s photograph was held on a local database and could not be searched against other photographs. However, the technology had developed since then, and the police is now able to apply facial recognition and facial mapping techniques to the applicant’s photograph by uploading it to a national database.
The potential use of facial recognition was influential in determining whether there had been an interference with the right to privacy. The Court also highlighted the risk of such evolving technologies, in relation to state powers. In this regard, the Court stresses the need to examine compliance with the right to privacy when “obscure powers” are vested in a state, “creating a risk of arbitrariness, especially where the technology available is continually becoming more sophisticated” (para. 88).
Because the applicant’s personal data has been retained indefinitely without consideration of the seriousness of his offence, the need for indefinite retention and without any real possibility of review, the Court held, unanimously, that there had been a violation of Article 8 (right to respect for private and family life) of the European Convention on Human Rights.
This judgment is especially relevant because it shows that blanket data retention policies without any safeguards breach the right to privacy of individuals, even when measures are considered to fall within the state’s discretion. This judgment could also impact ongoing discussions in the EU around future data retention legislation, as well as ongoing cases in the Court of Justice of the EU.
Technologies like automated decision-making, biometrics, and unpiloted drones are increasingly controlling migration and affecting millions of people on the move. This second blog post in our series on AI and migration highlights some of these uses, to show the very real impacts on people’s lives, exacerbated by a lack of meaningful governance and oversight mechanisms of these technological experiments.
What can happen to you in your migration journey when bots are involved?
Before the border: Want to eat? Get your eyes scanned!
Before you even cross a border, you will be interacting with various technologies. Unpiloted drones are surveilling the Mediterranean corridor under the guise of border control. However, if similar technologies, like the so-called “smart border” surveillance at the US borders, are any indication, this may lead to more people dying as they are seeking safety. Biometrics such as iris scanning are increasingly being rolled out in humanitarian settings – where refugees, on top of their already difficult living conditions, are required to get their eyes scanned in order to eat, for example. And now, not even the personal posts you intended to share only with your friends and family are safe – social media scraping to screen your immigration applications is becoming common practice.
What is happening with all this data? Various international organisations like the United Nations (UN) are using Big Data, or extremely large data sets, to predict population movements. However, data collection is not an apolitical exercise, especially when powerful actors like states or international organisations like the UN collect information on vulnerable populations. In an increasingly anti-immigrant global landscape, migration data has also been misrepresented for political ends, to affect the distribution of aid dollars and resources and support hardline anti-immigration policies.
What is also concerning is the growing role of the private sector in the collection, use, and storage of this data. The World Food Program recently signed a 45 million USD deal with Palantir Technologies, who recently joined the EU lobby register, the same company that has been widely criticised for providing technology that supports the detention and deportation programs run by US Immigration and Customs Enforcement (ICE). What will happen with the data of 92 million aid recipients shared with Palantir? What data accountability mechanisms are in place during this partnership, and can data subjects refuse to have their data shared?
At the border: Are you lying? A bot can tell!
When you arrive at the border, more and more machines have appeared to scan, surveil, and collect information about you. Increasingly, these machines rely on automated decision-making. However, instances of bias in automated decision-making are widely documented. Pilot projects have emerged to monitor your face for signs of lying, and if the system becomes more “skeptical” through a series of increasingly complicated questions, you will be selected for further screening by a human officer. However, can this system account for the cross-cultural differences in which we communicate? What about if you are traumatised and unable to recall details clearly? Discriminatory applications of facial or emotion recognition technologies has far reaching consequences on people’s lives and rights, particularly in the realms of migration.
Beyond the border: Automating decisions
When you are applying for a visa or want to sponsor your spouse to come join you, how do you feel about algorithms making decisions on your applications? A variety of countries have begun experimenting with automating decisions in immigration and refugee applications, visas, and even immigration detention. This use of technology may seem like a good idea, but many immigration applications are complicated. Already two human officers looking at the same set of evidence can make two completely different determinations. How will an automated system be able to deal with the nuances? Or what if you want to challenge in court a decision you don’t agree with? Right now it is unclear who is responsible for when things go wrong – is it the coder who creates the algorithm, the immigration officer using it, or even the algorithm itself?
Mistakes in migration decisions can have lasting repercussions – you can be separated from your family, lose your job, or even be wrongfully deported. This happened to over 7000 students who were deported from the UK based on a faulty algorithm that accused them of cheating on a language test. Where are these students now?
What about your human rights?
A number of your internationally protected rights are already engaged in the increasingly widespread use of new technologies that manage migration. These include equality rights and freedom from discrimination; life, liberty, and security of the person; freedom of expression; and privacy rights. When public entities are making decisions about you, your rights to due process are also affected, including a right to an impartial decision-maker, a right to appeal, and a right to know the case against you. These rights are particularly important to think about in a high-risk context such as when deciding on your unemployent benefits, whether or not you would be selected for a job interview, where the repercussions of getting decisions wrong can separate families, ruin career, or in the extreme circumstances, impact your life and liberty.
However, currently there is no integrated regulatory global governance framework for the use of automated technologies, and no specific regulations in the context of migration management. Much of the global conversation centres on ethics without clear enforceability mechanisms and meaningful accountability. The signing and ratification of Convention 108+ should be a priority for states around the globe, as well as the strong enforcement of the protection it envisages.
Our next blog post will explore some of these rights and suggests a way towards context-specific accountability and governance mechanisms for migration control technologies.
While automation is necessary for handling a vast amount of content shared by users, it makes mistakes that can be far-reaching for your rights and the well-being of society.
Most of us like to discuss our ideas and opinions on silly and serious issues, share happy and sad moments, and play together on the internet. And it’s a great thing. We all want to be free to learn about new things, get in touch with our friends, and reach out to new people. Every minute, photos, videos, and ideas are being shared. Every single minute, 527 760 photos are shared on Snapchat, 4 146,600 videos watched on YouTube, 456 000 tweets shared, and around 46 740 photos posted on Instagram. Do you know how many minutes we have in one day? 1440.
These pieces of information are different in nature. Some of them are home videos, and the law has nothing to do with them. But there is content that clearly breaches the law, such as child abuse material, or incitement to violence. And between legal and illegal content, there is a third group, which some people find harmful, while others have no problem with it. Although not illegal, but some parents would like to avoid their children getting access to pornography at the age of 12. It is not easy to define and let alone categorise what is harmful and for whom. It depends on culture, age, circumstances, and so many other factors.
Because a large quantity of internet content is hosted by online platforms, they have to rely on automated tools to find and tackle different categories of illegal or potentially harmful content. In particular, dominant players such as Facebook and Google have been using monitoring and filtering technologies for identification and removal of content. Do we agree on removing child abuse materials by allowing the police to investigate crimes? Certainly. Are we against permanent deletion of the content that would otherwise serve as vital evidence and documentation of gross human rights abuse as well as war crimes? Absolutely.
The EU, together with some Member States, has been continuously pushing online platforms to swiftly remove illegal or potentially harmful content, such as online hate speech or terrorism, often under the threat of fines if they don’t act fast enough. To meet these demands, tech companies have to rely on automated tools to filter out information that should not go online.
While automation is necessary for handling a vast amount of content shared by users, it makes mistakes that can be far-reaching for your rights and the well-being of society.
Contextual blindness of automated measures silences legitimate speech
Automated decision-making tools lack an understanding of linguistic or cultural differences. Content recognition technologies are unable to assess the context of expressions accurately. Even in straightforward cases, they make false matches. In 2017, the pop star Ariana Grande streamed her benefit concert “One Love Manchester” via her YouTube channel. The stream was promptly shut down by YouTube‘s upload filter, which wrongly flagged Grande’s show as a violation of her own copyright. On a more serious note, the same automated tools removed thousands of YouTube videos that could serve as evidence of atrocities committed against civilians in Syria, potentially jeopardising any future war crimes investigation that could bring war criminals to justice. Because of their contextual blindness or, in other words, inability to understand users’ real meaning and intentions, they flag and remove content that is completely legitimate. Thus, journalists, activists, comedians, artists, as well as any of us sharing our opinions and videos or pictures online risk being censored because internet companies are relying on these poorly working tools.
They’re not a silver bullet
These technologies are sometimes described as “Artificial Intelligence” (AI), a term that conjures up notions of superhuman computational intelligence. However, nothing of the sort exists, nor is it on the horizon. Instead, what this term refers to is advanced statistical models that have been trained to recognise patterns, but with no actual “understanding” or “intelligence”. Content recognition technologies cannot understand the meaning or intention of those who share a post on social media or the effect it has on others. They merely scan content for certain patterns such as visual, verbal, or audio files, which correspond to what they have been trained to identify as “hate speech” or “terrorist content”. There is no perfect, unambiguous training data, and so their ability to recognise these patterns is inherently limited to what they have been trained to recognise. Although they can achieve very high levels of accuracy in identifying unambiguous, consistent patterns, their ability to automate the very sensitive task of judging whether something constitutes hate speech will always be fundamentally limited.
Understandably, governments want to show their citizens that they are doing something to keep us safe from terrorism, hate speech, child abuse, or copyright breach. And companies are very happy to sell their automation technologies as a silver-bullet solution to politicians desperately digging for a simple answer. But we have to keep in mind that no automation will solve the problems deeply rooted in our society. We can use them as a tool to lessen the burden on platforms, but we need safeguards that ensure that we don’t sacrifice our human rights freedom because of poorly trained automated tools.
So who should decide what we see online? Read the next installment of this series in the next issue of the EDRi-gram newsletter.
The European Union has set digital transformation as one of its key pillars for the next five years. New data-driven technologies, including Artificial Intelligence (AI), offer societal benefits – but addressing their potential risks to our democratic values, the rule of law, and fundamental rights must be a top priority.
“By driving a human rights-centric digital agenda Europe has the opportunity to continue being the leading voice on data protection and privacy,” said Diego Naranjo, Head of Policy at European Digital Rights (EDRi). “This means ensuring fundamental rights protections for personal data processing and digitalisation, and a regulatory framework for governing the full lifecycle of AI applications.”
The EU must proactively ensure that regulatory frameworks (such as GDPR and the future ePrivacy Regulation) are implemented and enforced effectively. Where this doesn’t suffice, the EU and its Member States must ensure that the legislative ecosystem is “fit for the digital age”. This can be done by increasing the comprehensiveness (filling gaps and closing loopholes), clarity (clear interpretation), and transparency of EU and national rules. The principles of necessity and proportionality should always be front and centre whenever there is an inference with fundamental rights.
To deal with technological developments in a thorough way, in addition to data protection and privacy legislation, we need to take a look at other areas, such as competition rules and consumer law – including civil liability for harmful products or algorithms. Adopting a strong ePrivacy Regulation to ensure the privacy and confidentiality of our communications is also crucial.
From a fundamental rights perspective, one specific concern is the deployment of facial recognition technologies – whether AI-based or not.
“It is of utmost importance and urgency that the EU prevents the deployment of mass surveillance and identification technologies without fully understanding their impacts on people and their rights, and without ensuring that these systems are fully compliant with data protection and privacy law as well as all other fundamental rights,” said Naranjo.