Responding to a new European Union proposal for regulating the use of artificial intelligence technologies, Rasha Abdul Rahim, Director of Amnesty Tech, said:
“The EU’s proposal falls far short of what is needed to mitigate the vast abuse potential of technologies like facial recognition systems. Under the proposed ban, police will still be able to use non-live facial recognition software with CCTV cameras to track our every move, scraping images from social media accounts without people’s consent.
“While the proposal bans real-time facial recognition from being used by law enforcement in public spaces except under narrow circumstances, significant gaps remain in banning private uses of live facial recognition and many other forms of remote biometric surveillance.
“It also still allows real-time facial recognition to be used on people who are suspected of irregularly entering or living in a member state – this will undoubtedly be weaponized against migrants and refugees. The draft also has provisions allowing predictive policing and the use of AI systems in border control.
Under the proposed ban, police will still be able to use non-live facial recognition software with CCTV cameras to track our every move, scraping images from social media accounts without people’s consent
Rasha Abdul Rahim
“Overall, the proposed legislation does not go far enough in addressing the risks of AI entrenching and exacerbating racism and discrimination. Not only has research shown that facial recognition software is overwhelmingly less accurate with Black and Brown faces, but systemic racism in law enforcement means this technology can disproportionally be used against these communities and can lead to wrongful arrests. What’s more, the safeguards and transparency obligations outlined in the proposal will fail to meaningfully protect the public.
“The proposed ban on social scoring software, a dystopian technology which ‘ranks’ individuals according to reductive metrics, which can include gender, race, age and disability, is a welcome step. These AI systems are an affront to human dignity and can be used to target and exclude already marginalized groups.
“We call on the EU to close the many loopholes in this regulation which leave the door open to rampant abuse and discriminatory practices, including banning the use of all facial recognition systems used for mass surveillance.
Amnesty’s Ban the Scan campaign calls for a ban on the development, sale, deployment and export of facial recognition technologies for identification by state and private actors.”
Background
On Wednesday 21 April, the European Commission proposed legislation prohibiting certain uses of artificial intelligence systems, including the law enforcement’s use of ‘real-time’ remote biometric identification systems, such as facial recognition technologies, in public spaces.
However, such use would be permitted when “strictly necessary” for specific purposes and uses of biometric identification systems in other settings are not prohibited but only categorised as ‘high-risk’.