AI Act

Joint statement: The EU AI Act must protect people on the move

The European Union Artificial Intelligence Act (AI Act) will regulate the development and use of ‘high-risk’ AI, and aims to promote the uptake of ‘trustworthy AI’ whilst protecting the rights of people affected by AI systems.

However, in its original proposal, the EU AI Act does not adequately address and prevent the harms stemming from the use of AI in the migration context. Whilst states and institutions often promote AI in terms of benefits for wider society, for marginalised communities, and people on the move (namely migrants, asylum seekers and refugees), AI technologies fit into wider systems of over-surveillance, criminalisation, structural discrimination and violence.

It is critical that the EU AI Act protects all people from harmful uses of AI systems, regardless of their migration status. We, the undersigned organisations and individuals, call on the European Parliament, the European Commission, the Council of the European Union, and EU Member States to ensure the EU Artificial Intelligence Act protects the rights of all people, including people on the move. We recommend the following amendments to the AI act:

1. Prohibit unacceptable uses of AI systems in the context of migration

Some AI systems pose an ‘unacceptable risk’ to our fundamental rights, which will never be fixed by technical means or procedural safeguards. Whilst the proposed AI Act prohibits some uses of AI, it does not prevent some of the most harmful uses of AI in migration and border control, despite the potential for irreversible harm.

The AI Act must be amended to include the following as ‘prohibited practices’:

  • Predictive analytic systems when used to interdict, curtail and prevent migration. These systems generate predictions as to where there is a risk of “irregular migration” and are potentially used to facilitate preventative responses to forbid or halt movement, often carried out by third countries enlisted as gatekeepers of Europe’s borders. These systems risk being used for punitive and abusive border control policies that prevent people from seeking asylum, expose them to a risk of refoulement, violate their rights to free movement and present risks to the right to life, liberty, and security of the person.
  • Automated risk assessments and profiling systems. These systems involve the use of AI to assess whether people on the move present a ‘risk’ of unlawful activity or security threats. Such systems are inherently discriminatory, pre-judging people on the basis of factors outside of their control, or on discriminatory inferences based on their personal characteristics. Such practices therefore violate the right to equality and non-discrimination, the presumption of innocence and human dignity. They can also lead to unfair infringements on the rights to work, liberty (through unlawful detention), a fair trial, social protection, or health.
  • Emotion recognition and biometric categorisation systems. Systems such as AI ‘lie-detectors’ are pseudo-scientific technology claiming to infer emotions on the basis of biometric data, while behavioral analytics are used to detect ‘suspicious’ individuals on the basis of the way they look. Their use reinforces a process of racialised suspicion towards people on the move, and can automate discriminatory assumptions.
  • Remote Biometric Identification (RBI) at the borders and in and around detention facilities. A ban on remote biometric identification (such as the use of facial recognition) is required to prevent the dystopian scenario in which technologies are used to scan border areas as deterrence and part of a wider interdiction regime, preventing people from seeking asylum and undermining Member States’ obligations under international law, in particular upholding the right to non-refoulement.
2. Expand the list of high-risk systems used in migration

While the proposal already lists in Annex III the uses of ‘high-risk’ AI systems in migration and border control, it fails to capture all AI-based systems that affect people’s rights and that should be subject to oversight and transparency measures.

To ensure all AI systems used in migration are regulated, Annex III must be amended to include the following as ‘high-risk’:

  • Biometric identification systems. Biometric identification systems (such as mobile fingerprint scanners) are increasingly used to perform identity checks, both at and within EU borders. These systems facilitate and increase the unlawful and harmful practice of racial profiling, with race, ethnicity or skin  colour serving as a proxy for an individual’s migration status. Due to the severe risks of discrimination that come with the use of these systems, lawmakers must ensure the EU AI Act regulates their use.
  • AI systems for border monitoring and surveillance. In the absence of safe and regular pathways to the EU territory, people will cross European borders via irregular means. Authorities increasingly use AI systems for generalised and indiscriminate surveillance at borders, such as scanning drones or thermal cameras. The use of these technologies can exacerbate violence at the borders and facilitate collective expulsions or illegal pushbacks. Given the elevated risks and broader structural injustices, lawmakers should include all AI systems used for border surveillance within the scope of the AI Act.
  • Predictive analytic systems used in migration, asylum and border control. Systems used to generate predictions as to migration flows may have vast consequences for fundamental rights and access to international protection procedures. Often these systems influence how resources are assessed and allocated in the migration control and international protection contexts. Incorrect assessments about migration trends and reception needs will have significant consequences for the preparedness of Member States, but also for the likelihood that individuals can access international protection and numerous other fundamental rights. As such predictive systems should be considered as ‘high-risk’ when deployed in the context of migration. 
3. Ensure the AI Act applies to all high-risk systems in migration, including those in use as part of EU IT systems

Article 83 of the AI Act lays out the rules for AI systems already on the market, at the time of the legislation’s entry into force. Article 83 includes a carve-out for AI systems that form part of the EU’s large-scale IT systems used in migration, such as Eurodac, the Schengen Information System, and ETIAS. All of these large-scale IT systems – which foresee a capacity of over 300 million records – involve the automated processing of personal and sensitive data, automated risk assessment systems or the use of technology for biometric identification. For example, the EU plans to subject all visa and ‘travel authorisation’ applicants to automated risk profiling technologies in the next few years. Further, EU institutions are currently considering an update to Eurodac to include the processing of facial images in databases of asylum applicants. 

The exclusion of these databases would mean the safeguards in the EU AI Act do not apply. This blanket exemption will only serve to decrease accountability, transparency and oversight of AI systems used in EU migration control, and lessen protection for people impacted by AI systems as part of EU large-scale EU IT systems. Due to the exemption from regulatory scrutiny of these systems, the EU AI Act would lead to a double-standard when it comes to protecting fundamental rights of persons, depending on their migration status.

The EU AI Act should be amended to ensure that Art. 83 applies the same compliance rules for all high-risk systems and protects the fundamental rights of every person, regardless of their migration status.

4. Ensure transparency and oversight measures apply

People affected by high-risk AI systems need to be able to understand, challenge, and seek remedies when those systems violate their rights. In the context of migration, this requirement is both urgent and necessary given the overwhelming imbalance of power between those deploying AI systems and those subject to them.

The EU AI Act must prevent harm from AI systems used in migration and border control, guarantee public transparency, and empower people to seek justice. The EU AI Act must be amended to:

  • Include the obligation on users of high-risk AI systems to conduct and publish a fundamental rights impact assessment (FRIA) before deploying any high-risk AI system, as well as during its lifecycle.
  • Ensure a requirement for authorities to register the use of high-risk – and all public – uses of AI for migration, asylum and border management in the EU database. Public transparency is essential for effective oversight, particularly in the high risk areas of migration where a number of fundamental rights are at stake. It is crucial that the AI Act does not allow carve-outs for transparency measures in law enforcement and migration. 
  • Include rights and redress mechanisms to enable people and groups to understand, seek explanation, complain and achieve remedies when AI systems violate their rights. The AI act must provide effective avenues for affected people, or public interest organisations on their behalf, to challenge AI systems within its scope if they are non-compliant or violate fundamental rights.

Drafted by: Access Now, European Digital Rights (EDRi), Platform for International Cooperation on Undocumented Migrants (PICUM), and the Refugee Law Lab. With the support of: Amnesty International, Avaaz, Border Violence Monitoring Network (BVMN), EuroMed Rights, European Center for Not-for-Profit Law (ECNL), European Network Against Racism (ENAR), Homo Digitalis, Privacy International, Statewatch, Dr Derya Ozkul, Dr. Jan Tobias, and Dr Niovi Vavoula.

Signed by:
  1. Abolish Frontex, Europe
  2. Access Now, International
  3. Albanian Media Council, Albania
  4. AlgoRace, Spain
  5. Algorights, Spain 
  6. AlgorithmWatch, Germany
  7. All Faiths And None, United Kingdom
  8. Àltera APS, Italy
  9. Alternatif Bilisim (Alternative Informatics Association), Turkey
  10. Amnesty International, International
  11. ARCI, Italy
  12. Are You Syrious, European
  13. ARSIS Asociation for the Social Support of Youth, Greece
  14. Asociación Nacional Presencia Gitana, Spain
  15. Asociación Por Ti Mujer, Spain
  16. Aspiration, USA/International
  17. Association for Integration and Migration, Czech Republic
  18. Association for Juridical Studies on Immigration (ASGI), Italy
  19. ASTI Luxembourg – Association de soutien aux travailleurs immigrés, Luxembourg
  20. AsyLex, Switzerland
  21. Avaaz, International
  22. Baladre, Spain
  23. Bits of Freedom, Netherlands
  24. Blindspots, Germany
  25. Border Criminologies, United Kingdom
  26. Border Violence Monitoring Network, Europe
  27. Bürgerrechte & Polizei/ CILIP, Germany
  28. C.N.C.A. Coordinamento nazionale comunità accoglienti, Italy
  29. Catalina Quiroz-Niño, Spain
  30. CEDA – Center for Muslim Rights in Denmark, Denmark
  31. Center for AI and Digital Policy (CAIDP), International
  32. Center for Muslim Rights in Denmark (CEDA), Denmark
  33. Centre for Democracy & Technology, International
  34. Centre for Information Technology and Development (CITAD), Nigeria
  35. Centre for Peace Studies, Croatia
  36. Civil Liberties Union for Europe, European
  37. CNCD-11.11.11, Belgium
  38. Collective Aid, Bosnia Herzegovina
  39. Comisión Española de Ayuda al Refugiado (CEAR), Spain
  40. Comisión Legal Sol, Spain
  41. Comitato per i Diritti Civili delle Prostitute APS, Italy
  42. Comite de Apoyo a las Trabajadoras del sexo CATS, Spain
  43. Consortium for Refugees and Migrants in South Africa, South Africa
  44. Corporate Europe Observatory, Belgium
  45. Czech Helsinki Committee, Czech Republic
  46. D64 – Zentrum für Digitalen Fortschritt e. V., Germany
  47. Derechos Digitales, Latin America
  48. Digital Society, Switzerland
  49. Digitalcourage, Germany
  50. Diotima – Centre for gender rights & equality, Greece
  51. Državljan D / Citizen D, Slovenia
  52. Electronic Frontier Finland, Finland/European
  53. Elektronisk Forpost Norge (EFN), Norway
  54. Equipo del Decenio Afrodescendiente, Spain
  55. Erletxe, Spain
  56. Estonian Human Rights Centre, Estonia
  57. EuroMed Rights, Regional (Europe/Middle East/North Africa)
  58. European Anti poverty Network, European
  59. European Center for Human Rights, European
  60. European Center for Not-for-Profit Law (ECNL), Europe
  61. European Civic Forum, European
  62. European Council on Refugees and Exiles (ECRE), European
  63. European Digital Rights (EDRi), European
  64. European Federation of National Organisations Working with the Homeless (FEANTSA), European
  65. European Muslim Initiative for Social Cohesion, Denmark
  66. European Network Against Racism (ENAR), European
  67. European Network for the Promotion of Rights and Health among Migrant Sex Workers (TAMPEP), Netherlands
  68. European Network On Religion and Belief (ENORB), European
  69. European Network on Statelessness, European
  70. European Race and Imagery Foundation, Netherlands/Switzerland/Germany
  71. European Sex Workers’ Rights Alliance (ESWA), Europe/Central Asia
  72. Fair Trials, International
  73. FAIRWORK Belgium, Belgium
  74. Fem-R, Finland
  75. Feminist Autonomous Centre for Research, Athens, Greece
  76. Generation 2.0 for Rights, Equality & Diversity, Greece
  77. Glitch, United Kingdom
  78. Global Data Justice project (Tilburg Institute for Law, Technology and Society), Netherlands
  79. Greek Forum of Migrants, Greece
  80. Hacklab-ferro, Spain
  81. Haringey Welcome, United Kingdom 
  82. Health Action International, Netherlands
  83. Hermes Center, Italy
  84. Homo Digitalis, Greece
  85. HumanRights360, Greece
  86. I Have Rights, Greece
  87. In-Exile, Germany
  88. Institute Circle, Slovenia
  89. Institute for Strategic Dialogue (ISD), UK / Germany / International
  90. International Detention Coalition, International
  91. International Federation For Human Rights (FIDH), International
  92. International Federation of ACAT (FIACAT), France
  93. International Women* Space Berlin, Germany/European
  94. IRIDIA Cener of Defense of Human Rights, Spain
  95. Irish Council for Civil Liberties (ICCL), Ireland
  96. IT-Pol, Denmark
  97. Iuridicum Remedium, Czech Republic
  98. Ivorian Community of Greece, Greece
  99. Kif Kif vzw, Belgium
  100. KISA – Equality, Support, Antiracism, Cyprus
  101. Koapanang Africa Against Xenophobia {KAAX], South Africa/International
  102. KOK – German NGO Network against Trafficking in Human Beings, Germany
  103. La Strada International, Netherlands
  104. Lafede.cat, Spain
  105. Lassane Ouedraogo – Africa Solidarity Centre Ireland, Ireland
  106. Lawyers for Human RIghts, South Africa/International
  107. Legal Centre Lesvos, Greece
  108. Ligue Des Droits De L’Homme, France
  109. Ligue des Droits Humains, Belgium
  110. Migrant Tales, Finland
  111. Migrant Women Association Malta, Malta
  112. Migrants Organise, United Kingdom
  113. Migration-Controle.info, Germany/International
  114. Migreurop, France
  115. Mobile Info Team, Greece
  116. Moje Państwo Foundation, Poland
  117. Movement of Asylum Seekers in Ireland (MASI), Ireland
  118. NGO Legis, Republic of North Macedonia
  119. Novact, Spain
  120. Open Rights Group, United Kingdom
  121. Open Society Foundations, International
  122. Panoptykon Foundation, Poland 
  123. Platform for International Cooperation on Undocumented Migrants (PICUM), International
  124. Politiscope, Croatia
  125. Privacy International, International
  126. Privacy Network, Italy
  127. Prostitution Information Center (PIC), Netherlands
  128. R3D: Red en Defensa de los Derechos Digitales, Mexico
  129. Racism and Technology Center, Netherlands
  130. Red de Mujeres Latinoamericanas y del Caribe, Spain
  131. R3D: Red en Defensa de los Derechos Digitales
  132. Red Española De Inmigración Y Ayuda Al Refugiado, Spain
  133. Red Umbrella Athens, Greece
  134. Red Umbrella Sweden, Sweden
  135. Refugee Law Lab, York University, International
  136. Refugee Legal Support (RLS), Greece/United Kingdom/France
  137. Refugees International, United States
  138. Refugees Solidarity Movement, International
  139. Revibra Europe, European
  140. Salud por Derecho, Spain
  141. Samos Volunteers, Greece
  142. Sea-Watch, Germany/European 
  143. Sekswerkexpertise, Dutch Platform for the advancement of sex workers rights, Netherlands
  144. Sex Workers’ Empowerment Network, Greece
  145. SHARE Foundation, Serbia/South East Europe
  146. SOS Malta, Malta
  147. SOS Racismo Gipuzkoa, Spain
  148. STAR-STAR Skopje, North Macedonia
  149. Statewatch, European
  150. Stichting LOS, Netherlands
  151. Still I Rise, International
  152. StraLi for Strategic Litigation , Italy
  153. Subjective Values Foundation, Hungary
  154. SUPERRR Lab, Germany
  155. SW Digitaal, Netherlands
  156. Symbiosis – School of Political Studies in Greece, Greece
  157. Tamkeen for Legal aid and Human Rights, Jordan
  158. Taraaz, International
  159. The App Drivers and Couriers Union, United Kingdom
  160. The Joint Council for the Welfare of Immigrants, United Kingdom
  161. Today is a New Day, Institute for Other Studies, Slovenia
  162. Trans United Europe/BPOC Trans network, Netherlands
  163. Utrecht University, Digital Migration Special Interest Group, Netherlands
  164. Waterford Integration Services, Ireland
  165. Yoga and Sport With Refugees , Greece

Individuals:

  1. Angela Daly, Professor of Law & Technology, Leverhulme Research Centre for Forensic Science, Dundee Law School and International Research Fellow, Information Society Law Center, University of Milan, Italy
  2. Asli Telli, Research Associate at WISER, The Social and Economic Research Institute at Wits University
  3. Chiara De Capitani, PhD researcher in International Law at the Università di Napoli L’Orientale
  4. Claudia Aradau, Professor of International Politics, Department of War Studies and Principal Investigator of the ERC Consolidator Grant Security Flows,  King’s College London
  5. Cristina Del Biaggio
  6. Douwe Korff, Emeritus Professor of International Law, London Metropolitan University and Associate, Oxford Martin School, University of Oxford
  7. Dr Arjumand Bano Kazmi
  8. Dr Dale T McKinley, Senior Research Associate’ in the Department of Anthropology & Development Studies at the University of Johannesburg
  9. Dr Derya Ozkul, Senior Research Fellow, Refugee Studies Centre, University of Oxford
  10. Dr Grace S. Thomson
  11. Dr Koen Leurs, Associate Professor , Department of Media and Culture, Utrecht University
  12. Dr Niovi Vavoula, Lecturer (Assistant Professor) in Migration and Security at Queen Mary University of London
  13. Dr Philipp Seuferling, LSE Fellow, Department of Media and Communications, London School of Economics and Political Science
  14. Dr Sarah Perret, Research Associate, Department of War Studies, King’s College London
  15. Dr. Jan Tobias Muehlberg imec-DistriNet, KU Leuven, Belgium
  16. Edward Hasbrouck
  17. Elisa Elhadj, PhD Candidate KU Leuven & Research Fellow at the Center for AI and Digital Policy
  18. Francesca Meloni, Lecturer in Social Justice at the School of Education, Communication & Society, King’s College London
  19. Judith Membrives i Llorens, Universitat Oberta de Catalunya
  20. Luis H. Porras, Universidad Complutense de Madrid
  21. Marc Bria Rmírez, Universitat Autònoma de Barcelona
  22. Mary Gitahi, RLO study Uganda Lead Researcher, Refugee Studies Centre, University of Oxford
  23. Mirjam Twigt, Postdoctoral Fellow IKRS, Faculty of Law, University of Oslo
  24. Nabila Hamza
  25. Prof. Markus Krebsz, The Human-AI.Institute
  26. Theodora Christou, Queen Mary University of London
  27. Trivik Verma, Assistant Professor, Faculty of Technology, Policy and Management in Delft University of Technology
  28. Vicki Squire, Professor of International Politics, Department of Politics and International Studies, The University of Warwick
  29. Victoria Canning, Associate Professor of Criminology, University of Bristol
  30. Yassine Boubout