Trapped by Automation: Poverty and discrimination in Serbia’s welfare state

Executive Summary

In March 2023, Bogdan tried to renew the social assistance that he and his family, including four young children, depended on. Instead of receiving the assistance they so desperately needed, he was told that he had “too much money” in his bank account. “This was ridiculous. I don’t even have a bank account, and 90,000 Serbian dinars (770 euros) is a huge amount of money,” he said. “I tried to get information from the Centre for Social Work about this bank [account], but they were not able to tell me. They said it was my job to get the paperwork confirming that this was not correct.” When Amnesty International interviewed him, Bogdan was trying to collect the proof that he had no bank account nor the stated funds to his name, so that he could reapply for social assistance, without which Bogdan’s family was in severe crisis.  “We always lived modestly and without any luxuries,” but now we can’t even buy food,” Bogdan told Amnesty International.

Bogdan was one of possibly thousands of people who lost social assistance after the Social Card law entered into force in March 2022 and introduced automation into the process of determining people’s eligibility for various social assistance programmes.

A backbone of the Social Card law is the Social Card registry, a comprehensive centralized information system which uses automation to consolidate the personal and other data of applicants and recipients of social assistance from a range of official government databases. As well as data aggregation, the registry introduces semi-automated decision-making into the assessment of eligibility for social assistance and flags cases requiring review by a social worker. Of most interest for this research is when the registry creates a red flag notification in cases where it identifies a discrepancy between the data it has aggregated and the eligibility criteria to trigger an urgent review by a social worker.

Although the Serbian government presented the Social Card law as a tool to enable a fairer and more efficient delivery of social assistance, the law has had the opposite effect. Introducing technology into an already inadequate social assistance system has, in fact, exacerbated pre-existing gaps and further restricted people’s access to social security. Though exact numbers on benefits denials specifically due to the new system are unclear, the government’s own figures show 176,000 recipients in the system as of August 2023, which is 35,000 fewer than in March 2022 when the law entered into force. This figure does not capture all the people who lost social assistance due to flaws in the Social Card law’s implementation, but points to a continuous decreasing trend in the number of people who receive social assistance. The steep reduction in the pool of recipients, coinciding with the introduction of the law, is in stark contrast to the stubbornly high rates of absolute poverty in Serbia, which stand at 7% of the overall population. This means that almost a half million people live under the poverty threshold of 12,500 Serbian dinars (106 euros) per month and are unable to meet their basic needs.

Until the introduction of the Social Card registry, applicants had to collect a vast number of documents to prove their eligibility for different social protection programmes. While the new automated system somewhat eased the administrative burden on applicants, it has introduced a range of new barriers to the uptake of social assistance, especially for marginalized communities, and pushed some people deeper into poverty by denying them critical support.

This report is part of Amnesty International’s broader research on the uses of automated or algorithmic technologies in the public sector and their implications on human rights. It is based on comprehensive desk research, a review of the case files of five individuals who were in the appeals process and in-depth interviews with 21 recipients of social assistance in Serbia, all of whom had lost support after the introduction of the Social Card law. Amnesty International also spoke with social workers, government officials, and civil society representatives.

An already inadequate social assistance system

Even prior to the introduction of the Social Card registry, Serbia’s social assistance system was not fit for purpose. Social assistance is only provided to individuals and households that meet strict eligibility requirements related to unemployment status, income and assets. Some of the requirements, including an extremely low income threshold of 11,445 Serbian dinars (97 euros) – that is even below the absolute poverty level – are so restrictive that they, combined with other conditions, leave around 250,000 people who are living in poverty outside of the social assistance coverage despite being in need. Those who qualify for social assistance receive a monthly amount that is not sufficient to keep them out of poverty. The amount of financial social assistance in 2023 was set at around 11,445 Serbian dinars (97 euros) per person, meaning it was both below the absolute poverty threshold and the minimum monthly wage that was fixed at 53,000 Serbian dinars (460 euros).

Role of the World Bank

The World Bank has long played a key role in the economic transition in the Western Balkans, including in Serbia, with particular focus on the comprehensive reforms of these countries’ social protection systems. The development of social registries has become central to the Bank’s efforts to promote more effective poverty-targeting in order to make social security systems “fairer, more sustainable, more effective and better able to respond to people’s needs.” In Serbia, the World Bank funded the establishment of the Social Card registry in 2021 as part of a larger 82,600,000-euro public sector loan.

Yet, despite its responsibility as a UN institution to ensure that its projects do not negatively affect people’s enjoyment of human rights, it is unclear if the Bank has, in this case, conducted a due diligence exercise during project planning and design to identify potential human rights risks and put in place adequate mitigation measures. Amnesty International formally reached out to the World Bank on five occasions but has not received a response at the time of the publication.

This report found that the introduction of the Social Card registry has had a negative impact on people’s rights to social security, equality and non-discrimination, remedy and information. These are rights that are grounded in key international and regional instruments that Serbia has ratified, including the International Covenant on Economic, Social and Cultural Rights (ICESCR), the International Covenant on Civil and Political Rights (ICCPR) and the European Convention on Human Rights (ECHR).

Serbia’s obligations on the right to social security include ensuring that social support is adequate in amount and duration so that everyone can realize their rights to family protection and assistance, an adequate standard of living and adequate access to healthcare. Prior to the introduction of the Social Card registry, neither the amount nor the duration of support was sufficient, and the new system has exacerbated this by swiftly and, at times, erroneously removing people from essential social security support. Moreover, the Social Card Law does not provide access to an adequate right to remedy as guaranteed under the ECHR or access to their right to information, which is recognized by the ICCPR.

Finally, the right to equality and non-discrimination is one of the cornerstones of the international human rights framework. Amnesty International’s research found that the introduction of the Social Card registry did not uphold Serbia’s obligations to not only prevent discrimination but also take positive measures to bring about substantive equality where everyone enjoys their rights to the same extent. In the context of social security, states must guard against both direct and indirect discrimination, including by addressing laws and practices that appear neutral at face value but have a disproportionate effect on marginalized groups.

Key findings

The findings of this report indicate that instead of addressing the gaps in the social assistance system, the Social Card registry operationalized the existing restrictive eligibility conditions and exacerbated exclusion, particularly harming Roma and people with disabilities. These findings are consistent with research documenting the harms of welfare automation elsewhere. When imposed on a welfare system without meaningful safeguards, automation often amounts to an intrusive means of eligibility monitoring and disproportionately harms already marginalized groups.

Inaccurate data

The Social Card registry often relies on inaccurate source data on the earnings and assets of recipients. In cases documented for this report, people lost social assistance because the source data collected by the registry was completely incorrect, misrepresenting their income status or attributing income to individuals that they had never had.

For example, in February of 2023, Mirjana suddenly lost her daughter and was unable to afford the cost of the funeral. A Serbian human rights organization deposited 20,000 Serbian dinars (around 170 euros) into Mirjana’s bank account in the form of a relief payment to help cover the funeral costs. This money was instantly flagged by the Social Card registry as income, and the local Centre for Social Work informed Mirjana that she no longer qualified for social assistance. In the same month, Mirjana lost both her daughter and the regular monthly social assistance support she heavily relied on. With free legal aid, Mirjana successfully appealed her removal on the second attempt. However, the process took months, during which time she had no financial support from the state.

Mirjana was one of several people whose data were classified incorrectly, leaving her without social assistance. In a system that so heavily relies on data to determine people’s eligibility, the accuracy of data is essential to ensure a correct and fair outcome. Yet, Serbian authorities have not done enough to ensure that all participating databases are up to standard and contain accurate information, resulting in the Social Card registry arbitrarily depriving people of benefits.

Problems with data integrity for marginalized communities

It was not only that some of the data in the Social Card registry was inaccurate or incorrectly classified, but the system’s heavy reliance on raw data on assets raises concerns about data integrity especially when they pertain to marginalized communities. Some people lost social assistance because they were found to have multiple cars registered to their name, even though the cars were sold for scrap metal years previously.  Across Serbia, scrap metal vendors often fail to transfer car titles to their name after they purchase them from Roma to avoid paying tax. In some cases, this resulted in the loss of social assistance. Whilst prior to the introduction of the Social Card registry social workers were able to exercise some level of discretion and understood that records on car ownership were likely to be unreliable, the new system dramatically reduced the possibility to consider the context and to conduct field assessments – instead, relying solely on data which often fail to capture the complexity of people’s lives.

In an interview with Amnesty International, Dinko who has a family of nine, including his wife, five children and his elderly parents, recounted how he was told by staff in the Centre for Social Work in April 2023 that his family could no longer receive social assistance because the Social Card registry showed that his father owned two cars. Dinko explained that the cars, registered in his father’s name in 2013 and 2016, were never in a driving condition and had been sold for scrap metal many years ago. Unfortunately, Dinko’s father never officially de-registered the cars because he was unaware that this was necessary or could affect social assistance.  As a result, Dinko’s family spent months in severe economic hardship as they not only lost social assistance but also accompanying support, such as access to the soup kitchen.

Automation bias?

Although the Social Card registry does not make decisions on eligibility for social assistance, Amnesty International interviewed recipients who said that the social workers ascribed decisions to the “new system” and insisted that they were not able to contradict or override notifications that were pointing to what was clearly an error in their experience. “There is nothing I can do, it’s the new system from Belgrade that decided,” was perhaps the most frequently documented response by social workers, as noted in the interviews. Combined with the lack of training for social workers on how to use the system as an aid rather than defer to its outputs and mitigate potential automation bias, this raises concerns that the Social Card law seems to have significantly changed the role of social workers who are no longer able or empowered to use their own judgement to assess whether eligibility data are accurate or accurately reflect the socio-economic conditions of applicants.

System only tracking changes that can lead to the loss of assistance

The Social Card registry’s design does not seem to allow new information which would ensure that recipients whose circumstances have worsened from the previous month will receive the full amount of social protection they are entitled to.  The Social Card registry is automatically updated periodically, yet it appears that it prioritizes tracking changes in household conditions, which result in the loss or reduction of social assistance rather than increased eligibility. For example, if the registry determines that a person has earned additional income over the past month, it will quickly flag them for investigation by a social worker, who will then likely remove them from social assistance or reduce the amount they receive. However, if the same person does not earn the same income in the following month, the registry will not immediately or automatically reinstate the person’s benefits as they are no longer in the system. While clearly beneficial for the government budget, this feature places an additional burden on recipients who have to reapply in order to reinstate their benefits. It is also at odds with the government’s objective to ensure a fairer delivery of social assistance and greater inclusion.

Social Card registry operationalizing existing restrictions to social assistance

Amnesty International interviewed people who lost social assistance after the Social Card registry flagged them for having an income that put them above the minimum threshold. Yet this income frequently referred to the modest amounts that people made by collecting and selling secondary raw materials to supplement low amounts of social protection, which were insufficient to meet their basic needs. These earnings were often so negligible and irregular that they could not have significantly changed people’s economic situation to put them above the poverty line.

While the existing social assistance system already penalizes people who try to bridge the gap between social assistance and the actual cost of living, the Social Card registry has made the data absolute, removing any discretion that social workers could exercise while assessing claims and resulting in a rapid loss of social assistance. 

Discriminatory effect

The Social Card registry risks disproportionately harming Roma, people with disabilities and other marginalized communities who are over-represented in the social assistance system and penalised more by data quality issues. The authorities presented the registry to Amnesty International as a neutral technical solution. However, in practice, it cannot be isolated from the social and historical context into which it is introduced. Instead of taking into account and alleviating the existing challenges facing these communities, the new system has created a new barrier to accessing support and reinforced structural discrimination.

The consolidation of raw and often unrepresentative data on marginalized communities subjects these communities to arbitrary assessment of eligibility. The process completely ignores the fact that some people cannot easily keep their administrative affairs and records in order and up to date due to a combination of precarious living conditions, low literacy or digital literacy, as well as mistrust of the authorities.

The authorities did not take any special measures to ensure that affected individuals and communities had information about how the new system operated and how it would affect eligibility conditions for social assistance. Introducing a data-intensive system without a prior information campaign targeting these groups and adequate safeguards against potential discrimination will inevitably harm some communities, like Roma. The failure of the authorities to identify, prevent and mitigate the potential new barriers to access may amount to indirect discrimination.

Lack of remedy

The length of time it takes for people to be reinstated to social assistance is in sharp contrast with the speed with which people lose benefits. The mechanism for appeals and review of cases does not provide for a meaningful and effective remedy.Individuals who lose social assistance have fifteen days to file an appeal, a timeframe that many people find too short to allow them to gather all the required supporting documentation needed to successfully file an appeal. If people miss the fifteen-day window, they are forced to wait three months before they can reapply for support, regardless of their eligibility or need for social assistance.  The opacity of how the Social Card registry works, combined with a lack of information on how a decision was made in their case, has a dissuasive effect on people lodging appeals.

Lack of transparency

The Ministry of Labour has not publicly shared how the Social Card registry works, nor has it provided access to any human rights risk assessment of the registry undertaken so far. Amnesty International believes a review of the cases that the Social Card registry has flagged for investigation since its introduction is necessary to ascertain whether the system is producing discriminatory outputs.   

Currently, given the lack of transparency, any analysis of the registry’s operation is not possible. This undermines Serbia’s commitment to the principle of transparency in public affairs and presents a barrier to ensuring effective accountability. Indeed, more details about the Social Card registry would also enable affected communities and broader civil society to monitor its operation, identify concerns in a timely manner, and exercise their right to remedy and evaluate the efficacy of the system. Greater transparency, in order to enable accountability, improvements and understanding of the system, would also help build trust in the system and its operation.

When approached for a response, the Serbian authorities rejected the findings of this report. Where relevant, their responses were reflected in the full text of the report.

Conclusion and recommendations

The introduction of new technologies, including semi-automated decision-making, must first be evaluated for their potential risks and harms.  This means adequate and robust human rights impact assessments throughout the lifecycle of the system, from design to deployment, but also a consultation with communities who will be affected by the system.  Any changes resulting from the system must be communicated in a clear and accessible way. The systems which cannot be deployed in accordance with human rights should not be implemented at all.

The full list of recommendations is available in the report.

Key recommendations to the Serbian government:

  • Urgently increase the amount of social assistance to ensure that it is sufficient to guarantee the right to an adequate standard of living;
  • Remove all restrictive and discriminatory eligibility criteria that prevent people living in extreme poverty from accessing social assistance, including raising the minimal income threshold to qualify for social assistance;
  • Ensure that social assistance applicants receive clear and accessible information about how decisions are made about their cases, how to appeal such decisions, and, ensure that applicants receive support in lodging appeals;
  • Implement independent monitoring and oversight of the Social Card registry;
  • Conduct a comprehensive outreach programme to affected marginalized communities who receive social assistance to inform them about eligibility conditions, procedures as well as the increased importance of good record keeping and its impact on eligibility assessment. 

Key Recommendations to the World Bank:   

  • Conduct and publicly share an independent assessment of the human rights impact of Serbia’s Social Card registry and provide suggested remedial measures in the instance of any human rights risks;
  • Conduct and publish audits of social registries funded by the World Bank and ensure that such audits assess the rate of exclusion; causes of such errors; mitigation measures; and access to remedy, i.e., ability to challenge eligibility decisions.

Methodology

This report investigates the human rights impact of Serbia’s introduction of the Social Card registry in its social protection system. It focuses specifically on the role of automation in determining people’s eligibility for social assistance programmes and the impact of automation on marginalized communities. It contributes to Amnesty International’s long-standing research on economic, social, and cultural rights (ESCR), and the importance of adequate social protection measures to shield people from unexpected economic shocks, such as those caused by the Covid-19 pandemic and the war in Ukraine. This report will complement Amnesty International’s ongoing research on the increasing use of automation in the public sector, particularly in the sphere of social protection and the World Bank’s role in social protection reforms globally. The findings of the report build on the legal opinion submitted by Amnesty International and seven other human rights organizations in support of the A11 – Initiative for Economic and Social Rights-led challenge to the Social Cards Law before Serbia’s Constitutional Court. 

Amnesty International carried out research for this report between November 2022 and September 2023. This involved extensive desk research and qualitative interviews with recipients of financial social assistance in Serbia, social workers and other staff in Centres for Social Work, representatives of civil society organizations that provide assistance to marginalized communities in the field of economic, social, and cultural rights, as well as government authorities responsible for social protection, and independent government agencies monitoring equality and data protection. 

During the course of the research, Amnesty International conducted three research visits to cities and towns in Serbia, including Belgrade, Kraljevo, Novi Sad and Apatin. This involved in-depth face-to-face interviews with 21 recipients of social assistance, including 10 men, 11 women, 15 members of the Roma community, seven people living with a disability or caring for someone,1 and four individuals aged over 50. The purpose of these interviews was to (i) highlight people’s experience of living in poverty, specifically their experiences with social assistance after the Social Card Law and the Social Card registry were introduced in March 2022, and (ii) understand the potential impact of introducing automation into social assistance eligibility testing. 

This research highlights the voices of people who are experiencing the intersection of one or more forms of marginalization and discrimination in Serbia. The accounts of recipients are corroborated by government statistics, reports by international and civil society organizations and comprehensive data from other organizations, including the World Bank, the UN Development Programme (UNDP), the UN Children’s Fund (UNICEF), UN Population Fund (UNFPA), as well as the Statistical Office of the Republic of Serbia. Amnesty International also conducted extensive desk research, including a review of legal and policy frameworks governing social security in Serbia, tender documentation for the design and implementation of the Social Card registry, the World Bank’s loan agreement with Serbia, government instructions and training materials on the Social Card registry for the Centres for Social Work, research from partner organizations, media outputs, academic articles, official statistic case files of five people who appealed decisions to remove their social assistance after the Social Card registry was introduced. 

In the absence of full documentation on the Social Card registry’s design and operation, which Serbian authorities declined to publish or make publicly available, Amnesty International has built an understanding of the Social Card registry’s operation from multiple sources. These include official correspondence and an in-person meeting with the Ministry of Labour, Employment, Veterans and Social Policy (herein: Ministry of Labour) tender documentation for the Social Card registry design, interviews with social welfare centre staff, information and documents received from affected individuals, and interviews with local organizations providing legal and other assistance to individuals who lost social assistance during their appeals processes. 

In the context of this report, automation refers to a data-driven system which uses technology to pull data from multiple databases into the Social Card registry and process said data to identify possible discrepancies in the data that could affect people’s eligibility for social assistance. Serbia’s Social Card registry is a semi-automated system that selects cases for review by social workers and allows them, at least in theory, to independently decide on their outcome. 

Interviews with recipients of social assistance were conducted either in Serbian or English, with interpretation when necessary. All interviewees were informed about the nature and purpose of our research and how the information provided would be used, and gave their written consent. As is standard in Amnesty International’s objective and impartial research practices, no incentives were given to interviewees in exchange for their accounts. 

Throughout the report, the names of recipients and social workers, and in some instances, identifying details such as place names, have been excluded and/or changed to protect people’s privacy and confidentiality. 

Amnesty International also interviewed eight representatives of civil society organizations, four social workers, and eight government officials. Following an official written exchange with the Ministry of Labour, in August 2023, we also shared our findings with the Ministry in November 2023 and sought their written response to specific allegations ahead of the publication of this report. The responses from the Ministry from the 30th of August and 24th of November are reflected, where relevant, in the text of the report. 

On five occasions, Amnesty International requested an interview with World Bank representatives in Serbia to discuss the Bank’s role in the Social Card registry design and development, as well as the outcomes of any due diligence exercise on the human rights impact of the registry. Amnesty International also requested a written response from the World Bank Serbia and World Bank’s Western Balkans Programme to specific questions relating to the Bank’s involvement in the Social Card registry. Finally, the findings of this report were shared with World Bank officials prior to its publication, but no response was received by the time of the publication.


Footnotes

[1] This includes individuals caring for children as well as other family members.

Chapter 1 – Human rights risks of introducing digital technologies in the social protection field

The digital welfare state

The digital welfare state is either already a reality or emerging in many countries across the globe. In these states, systems of social protection and assistance are increasingly driven by digital data and technologies that are used to automate, predict, identify, surveil, detect, target and punish.

Phillip Alston, Former UN Special Rapporteur on Extreme Poverty and Human Rights

States often justify the digital transformation of social protection schemes as a means to increase efficiency or to better target those in need of state support. These transformations can include introducing automated or algorithmic decision-making, digitizing application processes for social assistance, and creating digital databases to store and process personal data. 

Although technology in the public sector is often presented as objective and unbiased, it is virtually impossible to create a value-neutral technology or database that is free from bias. Technology is not introduced in a vacuum but into existing societies and specific social and political contexts with all their flaws. It, therefore, inevitably reflects the underlying biases and worldviews of the people who built it. Introducing technology into social protection systems can have potentially unpredictable and unintended consequences for individuals. Such impacts can also vary widely depending on whether those individuals are already subject to systemic and intersectional forms of discrimination and marginalization.  

To identify and mitigate any potential bias, discrimination, or human rights harm, governments and policymakers should fully understand both the context in which these systems are deployed and the existing power imbalances and inequalities that underpin such systems. As the UN Special Rapporteur on contemporary forms of racism notes, states “must address not only explicit racism and intolerance in the use and design of emerging digital technologies” but also “and just as seriously, indirect and structural forms of racial discrimination that result from the design and use of such technologies”. 

A growing digitization in social protection, which requires individuals to engage with digitized services, may pose an additional barrier to people exercising their rights. This is especially the case for those with lower levels of literacy or digital literacy, people with disabilities, people in poverty, as well as those who may lack any official documents and paperwork required to interact with these systems. The UN Special Rapporteur on extreme poverty and human rights found that the “digitization of welfare systems” can lead to “a narrowing of the beneficiary pool” as well as “the introduction of demanding and intrusive forms of conditionality.”2  According to Mike Zajko, there is a fundamental tension in the use of digital technologies in the public sector: on the one hand, “government benefits provide necessary assistance to those in need and may be used to promote equity”, but on the other, “the social sorting inherent in these processes also functions as a mechanism of exclusion, targeting, and oppression”. 

Finally, even when digitized systems are removed from public sector use, they can still leave traces in institutions and in people’s lives. This is often referred to as an “algorithmic imprint” – a situation whereby even after an algorithmic system is removed, the impact of its use continues well after its deployment is stopped.  

When researching fraud detection in childcare benefits payments in the Netherlands, Amnesty International found that many families who were erroneously flagged as having committed fraud were forced to pay back what were perceived as excessive or erroneous childcare payments. This pushed many families into serious financial difficulties, including debt and bankruptcies. Many people were evicted from their homes when they could no longer afford their rent. Some people also reported suffering serious stress, which impacted their mental health. The algorithmic system behind the discriminatory fraud detection was later rolled back by the Dutch government, and a scheme was put in place to compensate people with a fixed amount regardless of their individual assessments. However, the serious harms caused by the system will likely have long-lasting effects on people’s lives. 

Digital technologies are often developed and introduced in the public sector without the meaningful involvement of people who will be interacting with these new systems. As a result, they are not always suited to specific groups’ needs and realities and can themselves become barriers to people accessing their rights. To mitigate the potential human rights harms of technology and in order to develop technology that works within existing complex social realities, states must incorporate diverse and representative perspectives and expertise in all stages of the planning and deployment of these systems.3 

Footnotes

[2] UN Special Rapporteur on extreme poverty and human rights, Report: Digital technology, social protection and human rights, 1 October 2019, UN Doc. A/74/493, para 5. See also Valery Gantchev, “Data protection in the age of welfare conditionality: Respect for basic rights or a race to the bottom?”, 22 March 2019, European Journal of Social Security, 21(1), https://doi.org/10.1177/1388262719838109, pp. 3-22.

[3] UN Special Rapporteur on contemporary forms of racism, racial discrimination, xenophobia and related intolerance, Report: Racial discrimination and emerging digital technologies: a human rights analysis, 18 June 2020, UN Doc. A/HRC/44/57, para. 17. Para. 14 also says: “Governments and the private sector must commit to approaches that include experts on the political, economic and social dimensions of racial discrimination at all stages of research, debate and decision-making to mitigate racially discriminatory design and use of emerging digital technologies. Affected racial and ethnic minority communities must play decision-making roles in the relevant processes.”


Automation and social protection

The increasing use of automation and algorithmic decision-making in all spheres of public and private life is threatening to disrupt the very concept of human rights as protective shields against state interference. The traditional asymmetry of power and information between state structures and human beings is shifting towards an asymmetry of power and information between operators of algorithms (who may be public or private) and those who are acted upon and governed.

Council of Europe

Automation is often used in social protection systems, policing, and border security and control, often with the stated purpose of increasing efficiency and decreasing costs. In the context of social protection, states have often argued that technology would help more accurately identify and prioritize people in need in order to better target the often-limited funding for social assistance available in national budgets.4 

Automation refers to a set of predefined instructions or tasks performed by a machine or technology. Automation is often used to streamline processes and tasks to support the human decision-maker and can include retrieving data from other databases or performing basic calculations. In social protection, some systems will rely on fully automated decision-making to, for example, determine people’s eligibility for social protection, whereby no human is involved in the decision-making process, and a decision is taken solely by the system. Other systems will use semi-automated decision-making, where a human is involved in the decision-making process in some capacity, often to review cases selected by the system. Semi-automated decision-making systems can also assist human decision-making processes by providing information and/or suggested outcomes. These systems frequently involve data-intensive solutions, such as creating social registries that collect and analyse vast amounts of information about recipients to determine their eligibility for social assistance programmes.5 

While states often present these technical innovations in social protection as “altruistic and noble enterprises” designed to improve governance and enhance access to benefits, they have often had the opposite effect. Indeed, the introduction of automation in social protection has often been accompanied by reduced budgets and the elimination of some services, leading to a reduction in the number of recipients, with women, racial and ethnic minorities, and people with disabilities disproportionally affected.  

Even systems where a human has a prominent role and can verify the output of an automated or semi-automated decision-making system can be potentially flawed. First, independent human decision-making can be impeded by so-called automation bias. This is a phenomenon whereby people tend to trust the automated system so much that they ignore other sources of information, including their own judgement, which can lead to errors within the automated decision-making not being detected or challenged.6 Second, automation bias compounds the human decision-makers’ existing biases and prejudices. Finally, even when a semi-automated decision-making system relies on a human to make the final decision, technology can still influence the decision-making process, including which cases are flagged by the system and, therefore, processed by the human case worker or which data is presented to inform human decision-making. 

Introducing automated or semi-automated decision-making to conduct eligibility assessments for social protection, perhaps the most common use of automation in welfare, has also raised the potential to drastically change the work of caseworkers who are managing claims. In Ontario, Canada, for example, the UN Special Rapporteur on extreme poverty and human rights reported that an automated system used to assess eligibility for social assistance produced 1,132 errors in either eligibility determinations or payments. This new system “reportedly led caseworkers to resort to subterfuge to ensure that beneficiaries were fairly treated; it also made decisions very difficult to understand and created significant additional work for staff.” 

A large body of work on the digital welfare state has shown that automated systems often exacerbate inequality,26 entrench discrimination,27 and pose a risk to human rights.28 Virginia Eubanks argues that “automated decision-making shatters the social safety net, criminalizes the poor, intensifies discrimination.”29 The UN Special Rapporteur on contemporary forms of racism has warned that when digitalization of welfare systems takes place in societies where groups are marginalized, discriminated against and excluded on a racial and ethnic basis, these systems are almost guaranteed to reinforce these inequities.”30 Indeed, systems using automation typically tend to be first deployed in the areas where the state intersects with low-income and marginalized groups, such as welfare systems. This often makes their deployment inherently discriminatory, as they tend to disproportionately target and profile marginalized populations.31 

Amnesty International’s research has shown that using automation, especially without adequate safeguards, can lead to disproportionate harm for already marginalized groups because of automation’s tendency to reproduce existing structural and systemic discrimination in societies – but at a greater scale.32 This happens because automation relies on using data that often reflects society’s historical and current state, including underlying values, assumptions and biases.33 In other words, even when automated systems do not directly or intentionally discriminate, they can often still have a disproportionate and discriminatory impact on marginalized groups.  

Given the extent and scope of risks posed to human rights by introducing digital technologies into the welfare state, it is crucial that states provide evidence that these systems are more accurate, better suited to the task, and do not violate human rights in order to justify their use. Furthermore, governments need to invest in improving digital literacy and accessibility across the board simultaneously by introducing technologies that rely heavily on digital access.  

Footnotes

[4] UN Special Rapporteur on extreme poverty and human rights, Report: Digital welfare states and human rights, 11 October 2019, UN Doc. A/74/493.; Mike Rutkowski, “Reimagining Social Protection”, International Monetary Fund, December 2018, https://www.imf.org/en/Publications/fandd/issues/2018/12/world-bank-reimagining-social-protection-rutkowski 

[5] “Entitlement to means-tested schemes is granted only to those with income or wealth below a prescribed threshold, as with minimum-income benefits, or to those meeting other criteria defined in proxy means tests.” United Nations, The Report on the World Social Situation 2018: Promoting Inclusion Through Social Protection, UN Doc. ST/ESA/366, p. 6.

[6] Saar Alon-Barkat and Madalina Busuioc, “Human–AI Interactions in Public Sector Decision Making: ‘Automation Bias’ and ‘Selective Adherence’ to Algorithmic Advice”, Journal of Public Administration Research and Theory, Volume 33, Issue 1, January 2023, https://doi.org/10.1093/jopart/muac007, pages 153–169. See also Algorithm Watch, “Poland: Government to scrap controversial unemployment scoring system”, 16 April 2019, https://algorithmwatch.org/en/poland-government-to-scrap-controversial-unemployment-scoring-system/  

[7] Amnesty International, Digitally Divided: Technology, inequality and human rights (Index: POL 40/7108/2023), 2 October 2023, https://www.amnesty.org/en/documents/pol40/7108/2023/en. See also Safiya Noble, “Algorithms of Oppression: How Search Engines reinforce Racism”, New York University Press, 2018; Virginia Eubanks, “Automating Inequality: How High-tech Tools Profile, Police and Punish the Poor”, 2018; Ruha Benjamin, “Race after Technology: Abolitionist Tools for the New Jim Code”, 2019; Rashida Richardson, “Racial Segregation and the Data-Driven Society: How Our Failure to Reckon with Root Causes Perpetuates Separate and Unequal Realities”, Berkeley Technology Law Journal no. 1051, 2022, p. 36.    

[8] Amnesty International, Digitally Divided: Technology, inequality and human rights (Index: POL 40/7108/2023), 2 October 2023, https://www.amnesty.org/en/documents/pol40/7108/2023/en. See also Safiya Noble, “Algorithms of Oppression: How Search Engines reinforce Racism”, New York University Press, 2018; Virginia Eubanks, “Automating Inequality: How High-tech Tools Profile, Police and Punish the Poor”, 2018; Ruha Benjamin, “Race after Technology: Abolitionist Tools for the New Jim Code”, 2019; Rashida Richardson, “Racial Segregation and the Data-Driven Society: How Our Failure to Reckon with Root Causes Perpetuates Separate and Unequal Realities”, Berkeley Technology Law Journal no. 1051, 2022, p. 36.  


The use of data

The “datafication” of individuals’ lives, where vast amounts of personal data are collected and processed, is a feature of many digital welfare states. It requires recipients to provide or consent to their personal data being collected and then used to determine their eligibility for welfare benefits. As the UN Special Rapporteur on extreme poverty and human rights has noted, this process creates serious risks because it effectively forces people to give up their right to privacy and data protection to exercise their right to social security. 

Processing data to assess eligibility for state support is not new. Today’s digital technologies have analogue predecessors that were used to categorize and assess people applying for welfare. However, the scale and breadth of data now being processed by digital systems, and the speed at which it is processed is new and can bring with it unintended and far-reaching consequences

The extensive amount of data being collected and processed by states as part of the digital welfare state raises concerns about the introduction of intrusive forms of government surveillance and a threat to the right to privacy. Surveillance could be defined as a system that includes “close monitoring, observation, record keeping, and categorization of information about individuals.” Welfare surveillance can be understood as using these means to monitor and track applicants and recipients of social protection. Welfare regimes that introduce a “means test” to determine an individual’s eligibility for social assistance are particularly prone to welfare surveillance. This is because, to ensure recipients’ eligibility for social protection, individuals are subject to continuous and ongoing scrutiny. This can include continuous examination of people’s financial records, marital or housing status, as well as – in the case of disability-related benefits – their medical records.9 Digital welfare surveillance, meaning welfare surveillance supported by technology, can often exacerbate discrimination felt by individuals and communities who were often already subjected to intense scrutiny before these new systems were introduced. Creating social registries that compile extensive profiles on recipients based on vast amounts of data, including personal data, only heightens the risks to the right to privacy. 

In parallel to this over-scrutiny, marginalized groups are often rendered invisible due to poor data collection practices that do not reflect their reality. For example, data collected on gender often does not allow people to identify as non-binary.10 Such poor data collection practices that neglect the realities of people and, in particular, of marginalized groups can lead to policy making that ignores their needs, as well as the introduction of data intensive systems based on inaccurate and unreliable data. 

Choices about which data is collected, how it is deemed to be important – or not – and how it is processed are not neutral decisions but rather reflective of states’ policy priorities. Because automated or semi-automated systems rely on data to function, the data needs to be both accurate and up to date. However, this can be very difficult to ensure when it comes to marginalized groups.11 This is because some marginalized people may face additional barriers to keeping their administrative affairs in order. These may include living in informal settlements and not having a recognized address to register with authorities or for correspondence, challenges with literacy that make completing forms difficult, as well as precarious or informal employment that does not provide accurate proof of earnings. 

A challenge to data accuracy, particularly in the context of social registries, is that data collected to determine people’s eligibility for social assistance creates a static snapshot of their socio-economic conditions that can quickly become outdated. “These technologies are developed to make decisions about people in the present and future on the basis of vast datasets collected, extracted, and translated from the past”. A social registry is only able to create a static and potentially inaccurate snapshot of an individual or household’s situation at a given point in time using the data that it can access, some of which might not accurately capture the complexity of people’s lives and their economic condition. However, “in the real world, even one specific characteristic of a household can change rapidly,” thereby creating an impossible situation of trying to establish a static representation of a household that may be constantly changing

As a solution for the static and potentially outdated data, some governments have introduced dynamic or integrated social registries which allow for regular data exchange across government institutions to conduct eligibility assessments for multiple social programmes. A key feature of a dynamic social registry is that, in theory, people can register and apply for a social programme when they need it, allowing the government to better respond to life cycle changes, such as job loss, disability or unpredicted economic or other shocks. 

However, dynamic social registries also entail numerous risks to people’s right to social security. Studies have shown that in countries that have adopted dynamic registries, social protection programmes had considerably lower reach and coverage of the eligible population, in large part due to traditional challenges that typically prevent marginalized and vulnerable households from accessing government programmes. Lack of information about how these programmes work, complexities of the application process and social barriers, including lack of trust in institutions, tend to significantly impede uptake in comparison to countries that do not use social registries or that rely on static information, such as census, to collect data on recipients. Furthermore, whilst, in theory, dynamic social registries allow people to move in and out of the social benefits system, once they are no longer eligible for support, applicants are often removed from the registry and have to reapply in order to receive support. This can present a significant barrier for many. Dynamic social registries also rely on processing of often significant amounts of personal data, which necessitates a robust legal and regulatory framework for data protection to be in place to prevent possible breaches of privacy as well as the need for the data itself to be accurate for it to be used in decision making.

Footnotes

[9] See Powell, Robyn, “Under the Watchful Eye of All: Disabled Parents and the Family Policing System’s Web of Surveillance”, 23 August 2023, 112 California Law Review (forthcoming), https://ssrn.com/abstract=4555846 or https://dx.doi.org/10.2139/ssrn.4555846 and Mike Zajko, “Automated Government Benefits and Welfare Surveillance” https://ojs.library.queensu.ca/index.php/surveillance-and-society/article/view/16107/10897 

[10] ESCR-Network, “Collective position data for Economic, Social, and Cultural Rights”, 28 April 2022, https://www.escr-net.org/news/2022/collective-position-data-economic-social-and-cultural-rights. See also European Commission, High Level Group on Non-discrimination, Equality and Diversity, Subgroup on equality data, “Guidance note on the collection and use of data for LGBTIQ equality”, 2023, https://commission.europa.eu/system/files/2023-07/JUST_Guidance%20note%20on%20the%20collection%20and%20use%20of%20data%20for%20LGBTIQ%20equality%20–%202023.pdf

[11] ESCR-Network, “Collective position data for Economic, Social, and Cultural Rights”, 28 April 2022, https://www.escr-net.org/news/2022/collective-position-data-economic-social-and-cultural-rights; ESCR-Net, Legal Opinion on International and Comparative Human Rights Law Concerning the Matter of the Social Card Law Pending before the Constitutional Court of Serbia, https://www.errc.org/uploads/upload_en/file/5435_file1_social-cards-legal-opinion–final-english-pub.pdf  

Given the potentially serious implications of automation in social security systems, it is not only important to gain a better understanding of how such technologies work but also to consider the responsibility of states and other actors, such as international organizations, as they pursue automation in the process of assessing eligibility for social assistance support. This section summarizes the applicable human rights framework related to some of the potential threats to human rights by the automation of social assistance outlined in this report, including: 

  • Right to social security. 
  • Right to equality and non-discrimination. 
  • Data protection frameworks. 
  • Right to privacy. 
  • Right to remedy. 
  • Right to information. 

The right to social security

The right to social security is recognized and protected by international human rights law. Article 9 of the International Covenant on Economic, Social and Cultural Rights (ICESCR) and Article 22 of the Universal Declaration of Human Rights (UDHR) recognize the right of everyone to social security. According to ICESCR, states are responsible for ensuring that social support is adequate in amount and duration so that everyone can realize their rights to family protection and assistance, an adequate standard of living and adequate access to healthcare.12 At a minimum, social assistance should secure protection against poverty, vulnerability, and social exclusion, enable people to live in health and dignity, and provide people with means that correspond to national poverty lines. 

Social assistance and broader programmes such as school meals and free textbooks are essential to realizing people’s economic, social, and cultural rights, including the rights to food, housing, water, education, and health. The Committee on Economic, Social and Cultural Rights (CESCR) has recognized that the right to social security is “of central importance in guaranteeing human dignity”13 and is an essential precondition of the right to an adequate standard of living and other rights, including the right to adequate food.14 States have an obligation to ensure the satisfaction of “minimum essential levels of benefits to all individuals and families that will enable them to acquire at least essential health care, basic shelter and housing, water and sanitation, foodstuffs, and the most basic forms of education”.15

Footnotes

[12] See UN CESCR, General Comment 19, 4 February 2008, para. 22; ICCPR, 1966, Article 9; UDHR, 1948, Article 22; European Social Charter (Revised) 1996, Articles 12, 23, and 30. 

[13] UN CESCR, General Comment 19: The Right to Social Security, 8 February 2008, UN Doc. E/C.12/GC/19, para. 1. 

[14] Article 11(1) ICESCR states that “The States Parties to the present Covenant recognize the right of everyone to an adequate standard of living for himself and his family, including adequate food, clothing and housing, and to the continuous improvement of living conditions.” See also UN CESCR, General Comment 19: The Right to Social Security, 8 February 2008, UN Doc. E/C.12/GC/19 , para. 28. 

[15] UN CESCR, General Comment 19: The Right to Social Security, UN Doc. E/C.12/GC/19 (2008), para. 59. 


The right to equality and non-discrimination

The right to equality and non-discrimination is essential to the enjoyment of all human rights and is one of the cornerstones of the international human rights framework.16 The UN Human Rights Committee (HRC) defines discrimination as “any distinction, exclusion, restriction or preference, which is based on any ground such as race, colour, sex, language, religion, political or other opinion, national or social origin, property, birth or other status, and which has the purpose or effect of nullifying or impairing the recognition, enjoyment or exercise by all persons, on an equal footing, of all rights and freedoms.”17

The Committee on Economic, Social, and Cultural Rights (CESCR) finds that “[d]iscrimination undermines the fulfilment of economic, social and cultural rights for a significant proportion of the world’s population” and has established how the principle of non-discrimination extends to core rights in ICESCR including the right to social security and the rights of individuals and groups such as people with disabilities.18 States have the obligation to eliminate all forms of discrimination, including formal or substantive discrimination, as well as direct and indirect discrimination. 19

In contrast with formal discrimination, which refers to discrimination enshrined in laws and policies, substantive discrimination refers to the results and outcomes of laws that are not discriminatory per se but may maintain and even exacerbate existing inequalities and disadvantages of particular groups. 

Similarly, indirect discrimination refers to a situation when a practice, rule, policy or requirement is outwardly neutral but has an adverse impact on a particular group. According to the CESCR, indirect discrimination happens when states fail to “remove promptly obstacles which the State party is under a duty to remove in order to permit the immediate fulfilment of a right guaranteed by the Covenant.”20

In the context of social security, under international human rights law, states must not only eliminate direct and indirect discrimination in law and practice but also take special measures to protect the most marginalized groups in society.21 Specifically, states need to pay special attention to groups who traditionally face difficulties in exercising the right to social security, including minorities, people with disabilities, and internally displaced persons, in all stages of the design and implementation of social protection programmes.22 State obligations for preventing discrimination experienced by specific groups are included in treaties, including the International Convention on the Elimination of All Forms of Racial Discrimination (ICERD); the Convention on the Rights of Persons with Disabilities (CRPD); and the Convention on the Elimination of Violence Against Women (CEDAW).

Footnotes

[16] The right to non-discrimination is recognized in Article 2 of UDHR, Article 2 of the ICCPR, Article 2 of the ICESCR, Article 14 of the Convention for the Protection of Human Rights and Fundamental Freedoms (ECHR), and Article 14 European Social Charter Article E Part V. 

[17] HRC, General Comment 18: Non-discrimination (1989) UN Doc. HRI/GEN/1/Rev.1, para. 7. 

[18] UN CESCR, General Comment 20: Non-Discrimination in Economic, Social and Cultural Rights (Article 2, para. 2) UN Doc. E/C.12/GC/20 (2009), para. 1.

[19] UN CESCR, General Comment 20: Non-Discrimination in Economic, Social and Cultural Rights (Article 2, para. 2) UN Doc. E/C.12/GC/20 (2009). 

[20] UN CESCR, General Comment 19: The Right to Social Security (Article 9 of the Covenant), 4 February 2008, UN Doc. E/C.12/GC/19, para. 65.

[21] UN CESCR, General Comment 19, 4 February 2008, UN Doc. E/C.12/GC/19, paras 29-30.

[22] UN CESCR, General Comment 19, 4 February 2008, UN Doc. E/C.12/GC/19, para. 31.

 


Human rights and personal data protection

Data protection principles stem from international human rights standards regarding privacy; information and public participation; due process; and remedy. Data protection can be understood as a series of safeguards that are designed to protect personal information that “is collected, processed and stored by ’automated’ means or intended to be part of a filing system.” 

There are legal safeguards to protect individuals from the potential harms of introducing automation in public sector decision-making. The EU’s General Data Protection Regulation (GDPR) states that “data subjects shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.”23 Exceptions to this right include if the data controller is authorized by a law that “lays down suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests.”24 These safeguards “include as a minimum a way for the data subject to obtain human intervention, express their point of view, and contest the decision.”25 The Council of Europe (CoE) Committee of Ministers has adopted the position that “[t]he use of automated decision-making systems should preserve the autonomy of human intervention in the decision-making process.” 

Article 35 of the GDPR states that data controllers are required to carry out a Data Protection Impact Assessment (DPIA). While a DPIA examines data protection risks, it is not an appropriate tool for the assessment of human rights risks. The DPIA is limited in scope and focuses solely on the processing of personal data, not, for example, on the socio-economic circumstances in which the system is used.26

Protection of personal data is not a stand-alone right under the European Convention for the Protection of Human Rights and Fundamental Freedoms (ECHR); however, the European Court of Human Rights (ECtHR) has recognized the right to protection of personal data in relation to the right to privacy, as guaranteed under Article 8.27 

Footnotes

[23] General Data Protection Regulation (GDRP) or Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119/1. 

[24] General Data Protection Regulation (GDRP) or Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119/1 Article 22(2)(a). 

[25] General Data Protection Regulation (GDRP) or Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119/1.  

[26] See Committee of Ministers to Member States, Recommendation on the human rights impacts of algorithmic systems, 8 April 2020, recital 5.2; European Commission Independent High-Level Expert Group on Artificial Intelligence, Ethics Guidelines for Trustworthy AI, 8 April 2019, p. 15

[27] European Court of Human Rights, S. and Marper vs. the United Kingdom, Nos. 30562/04 and 30566/04 (2008). 


The right to privacy

The right to privacy is protected under international human rights instruments, including Article 12 of the UDHR and Article 17 of the ICCPR, which provide that no one should be subject to “arbitrary or unlawful interference” with their privacy, family, home or correspondence, and this should be protected by law.28 The right to privacy is also well covered in international treaties protecting the rights of specific groups, including: Article 16 of the International Convention on the Rights of the Child (CRC); Article 14 of the International Convention on the Protection of the Rights of All Migrant Workers and Members of Their Families; and Article 22 of the CRPD. There are also provisions for the right to privacy in regional human rights instruments, including in Article 8 of the ECHR. 

Both the ICCPR and the CRPD recognize data protection as a core component of the right to privacy.29 The UN High Commissioner for Human Rights found that “the right to privacy plays a pivotal role in the balance of power between the State and the individual and is a foundational right for a democratic society. Its importance for the enjoyment and exercise of other human rights online and offline in an increasingly data-centric world is growing”.30 Any interference with the right to privacy by the state must be legal, necessary and proportionate. 

Footnotes

[28] International Covenant on Civil and Political Rights, Article 17. 

[29] International Covenant on Civil and Political Rights, Article 17; International Convention on the Rights of Persons with Disabilities, Article 22.

[30] UN Office of the High Commissioner for Human Rights (OHCHR), Report: The Right to Privacy in a Digital Age, 13 September 2021, UN Doc. A/HRC/ 48/31, para. 6. 


The right to remedy

International human rights law and standards contain provisions guaranteeing individuals the right to an effective remedy, as well as the right to adequate redress and due process. The UDHR states that “[e]veryone has the right to an effective remedy by the competent national tribunals for acts violating the fundamental rights.”31 The CESCR notes that “appropriate means of redress, or remedies, must be available to any aggrieved individual or group” and that “appropriate means of ensuring governmental accountability must be put in place”.32

The CESCR also states that any “withdrawal, reduction or suspension of benefits should be limited and based on the grounds that are reasonable, subject to due process, and provided for in the national law.”33

The ICCPR calls on State parties “[t]o ensure that any person whose rights or freedoms as herein recognized are violated shall have an effective remedy, notwithstanding that the violation has been committed by persons acting in an official capacity.”34 Article 13 of the ECHR states that “[e]veryone whose rights and freedoms as set forth in [the] Convention are violated shall have an effective remedy before a national authority notwithstanding that the violation has been committed by persons acting in an official capacity.”35 This right has been examined by the ECtHR in relation to cases pertaining to the effectiveness of remedies to the right to privacy (Article 8) and the right to non-discrimination (Article 1 of Protocol No. 12).  

The CESCR has concluded thatappropriate means of redress, or remedies, must be available to any aggrieved individual or group, and appropriate means of ensuring governmental accountability must be put in place”.36 CESCR also states that “[a]ll victims of violations of the right to social security should be entitled to adequate reparation, including restitution, compensation, satisfaction or guarantees of non-repetition.”37  

In the context of data protection, Article 11 of the CoE Convention for the protection of individuals with regard to the processing of personal data or Convention 108+ includes a provision for data subjects to “have a remedy under Article 12 where his or her rights under this Convention have been violated.”38

Footnotes

[31] United Nations, Universal Declaration of Human Rights 1948, Article 8. 

[32] UN CESCR, General Comment 9: The domestic application of the Covenant, 3 December 1998, UN Doc. E/C.12/1998/24, para. 2

[33] UN Committee on Economic, Social and Cultural Rights, General Comment 19: The right to social security (Article 9 of the Covenant), 4 February 2008, E/C.12/GC/19, Para.24

[34] UN General Assembly, ICCPR, 1966, Treaty Series 999 (December), 171 Part II, Article 2. 3 (a).

[35] Council of Europe, European Convention on Human Rights, 1950, Council of Europe Treaty Series 005.

[36] UN CESCR, General Comment 9: The domestic application of the Covenant, 3 December 1998, UN Doc. E/C.12/1998/24, para. 2

[37] UN CESCR, General Comment 19: The Right to Social Security, 23 November 2007, UN Doc. E/C.12/GC/19, para. 77.

[38] In the context of the Social Card Law, as per the CoE definition of data subjects, this refers to individuals who are either applicants or recipients of social protection and are having their data used by the Social Card registry. Council of Europe, Convention 108+, Convention for the protection of individuals with regard to the processing of personal data, June 2018. 


Transparency and right to information

Transparency is a key principle of governance and should be at the core of creating and implementing public sector policies, including in cases concerning individuals’ access to public services such as social protection, and any instances of automation or data-driven processes using sensitive data. To comply with transparency principles, states have an obligation to ensure the general right of access to information held by public bodies and to create mechanisms to enable individuals to request and access information.39 The principle of transparency also extends to data protection. Articles 13 and 14 of the GDPR require that the processing of personal data be transparent.40 Article 8 of the CoE’s Convention 108+ lays out provisions for transparency that include information on how the data controller should inform data subjects of, for example, the legal basis and purpose of their data being processed, as well as the categories of personal data being processed. 

States must meet their obligations to provide clear, timely, and comprehensive information on public sector decision-making. The right to access information is a core part of the right to participation and freedom of expression.41 It is protected by ICCPR Article 19, as citizens need access to information in order to meaningfully participate in public affairs.42 The UN Human Rights Committee (HRC) stated that ICCPR Article 19 allows for the right to access information held by public bodies.43 The UN Special Rapporteur on freedom of opinion and expression, the Organization for Security and Co-operation in Europe (OSCE) Representative on Freedom of the Media and the Organization of American States (OAS) Special Rapporteur on Freedom of Expression found that “[t]he right to access information held by public authorities is a fundamental human right which should be given effect at the national level through comprehensive legislation” and that furthermore states should proactively make public “a range of information of public interest.”44

Footnotes

[39] UN HRC, General Comment 34, 12 September 2011, UN Doc. CCPR/C/GC/34.

[40] General Data Protection Regulation (GDRP) or Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119/1 Article 22(2)(a). 

[41] The right to participation is included in several human rights instruments, including UDHR Article 21 and ICCPR Articles 20 and 27.  

[42]  ICCPR, Article 19. 

[43] UN HRC, General Comment 34, Article 19: Freedoms of opinion and expression. 12 September 2011, UN Doc.  CCPR/C/GC/34

[44] The UN Special Rapporteur on Freedom of Opinion and Expression, the OSCE Representative on Freedom of the Media and the OAS Special Rapporteur on Freedom of Expression. International Mechanisms for Promoting Freedom of Expression, Joint Declaration, 6 December 2004, p. 2

An illustration of a  woman in a large field picking up rubbish
© Simina Popescu

Chapter 3 – The case of Serbia’s social card law

The social card registry

Serbia’s National Assembly adopted the Social Card Law in February 2021 as part of the country’s Programme of Economic Reforms 2020-2022. The law entered into force on the 1st of March 2022, and by September 2023, it had been rolled out in 167 cities and towns for use in nearly all Centres for Social Work across the country.45 The law established the Social Card registry – an integrated and centralized electronic database, which pulls data on recipients’ and applicants’ socio-economic status from a range of official databases, which is then used by social workers to assess their eligibility for social protection.46

The establishment of a Social Card registry that centralize data on social protection recipients and applicants has been a key priority for successive governments in Serbia since 2017. This also represented a flagship project for the Ministry of Labour, Employment, Veterans and Social Affairs (the Ministry of Labour), which noted that the purpose of the registry was to ensure a fairer distribution of resources, to make social assistance delivery more efficient and to reduce potential fraud. The overall stated goal was to ensure that social assistance reaches all those who meet the eligibility conditions.47

The development of the Social Card registry had strong advocates in the “International Community” in Serbia.48 In its Reform Programme recommendation, the European Union (EU) Employment and Social Reform Programme (ESRP) included references to greater information sharing and interconnection of information systems in different sectors as a solution in Serbia’s reform process necessary for EU accession. More significantly, the World Bank directly supported Serbia’s development of its Social Card registry by providing expert guidance and financial assistance in the form of a long-term loan. This was part of its broader global programme supporting the establishment of such registries in the social protection field.49 The World Bank has supported or promoted similar data-driven social registry databases elsewhere, including in Jordan, Lebanon, Haiti, Nigeria, Morocco and Angola, as well as in Montenegro and Bosnia and Herzegovina.50

While designing the Social Card registry, the Serbian government collaborated closely with the Danish government to replicate parts of the Danish Social Card model.51 In 2017, Serbia’s former Minister of Labour, Employment Veterans and Social Affairs touted the Danish model as the “perfect system”, allowing the authorities to automatically recognize people in need of social assistance and eliminate lengthy application processes. However, the Danish model of social security is more comprehensive than only the Social Card registry and may not have been entirely appropriate for Serbia, whose social assistance programmes already suffer from persistent underfunding and low reach.52 

Denmark’s social security system has indeed been hailed by some as expansive and generous for decades. However, it has also faced increasing criticism for using potentially intrusive and discriminatory technology to determine people’s eligibility for social protection. Due to a highly polarized national debate on the scope of its social welfare system, Danish authorities have more recently shifted focus from its famously generous and broad coverage for everyone in need to addressing what is perceived as widespread welfare fraud. A key component of fraud detection has been the introduction of extensive databases collecting information about welfare recipients and other related persons (immediate relatives and, in some cases, partners and ex-partners), in turn laying the pipeline for data-driven automation systems. By establishing such intrusive means to determine eligibility for benefits, the Danish system has faced charges of conducting “massive surveillance” and violating recipients’ privacy rights, as well as introducing the risk of discrimination and exclusion. 

Nevertheless, in conversations with Amnesty International, Serbian authorities maintained that they considered the Danish welfare system to be a good example to follow.53 They have modelled key aspects of the Social Card registry of the Danish system, including the automated dynamic social registry to determine applicants’ socio-economic status and cross-check eligibility for different types of social assistance. 

Footnotes

[45] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[46] Serbia’s Social Card Law, Article 3

[47] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International

[48] In the countries of the former Yugoslavia, the term “International Community” describes the collection of international actors which collectively and individually play an active role or support the governance in the country. These include the United Nations (UN) and its agencies, international financial institutions such as the World Bank and International Monetary Fund (IMF), regional organizations such as the Organization for Security and Cooperation in Europe (OSCE) and the CoE, the EU delegation, and diplomatic missions of the US and Western European countries. Either due to political influence or funding incentives, the International Community tends to have a substantial influence on policy development and practice in these countries.  

[49] Public Sector Efficiency and Green Recovery Development Policy Loan agreement between Republic of Serbia and International Bank for Reconstruction and Development, https://documents1.worldbank.org/curated/en/908181627498690315/pdf/Official-Documents-Loan-Agreement-for-Loan-9235-YF.pdf. The loan was a part of the World Bank’s support for Western Balkan countries taking steps to “improve their social protection systems to make them fairer, more sustainable, more effective and better able to respond to people’s needs.” See: https://www.worldbank.org/en/region/eca/brief/advancing-social-protection-and-opportunities-for-reform-in-the-western-balkans  

[50] The development of the social registry in Republika Srpska in Bosnia and Herzegovina is still in the planning stages. 

[51]  Statement on Serbia’s Ministry of Labour, Employment, Veterans and Social Affairs website, https://www.minrzs.gov.rs/sr/projekti/prioriteti/socijalne-karte. See also Blic, “Danska pomaže Srbiji da uvede socialne karte,” 30 April 2018, https://www.blic.rs/vesti/drustvo/danska-pomaze-srbiji-da-uvede-socijalne-karte/c2q10fv 

[52] Denmark has one of the most generous welfare systems in the world, with 26% of the country’s GDP spent on benefits (in contrast to Serbia’s less than 3% of GDP going to non-contributory benefits despite the higher number of people living in absolute poverty). Moreover, in addition to the social registry that has access to multiple databases containing extensive information of people, the Danish system also includes a sophisticated and high-tech fraud detection system, and uses AI to analyze data and predict potential fraud. It is because of this that Danish authorities have been accused of overreach and subjecting recipients to mass surveillance by the state.

[53] Meeting with representatives of Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 28 April 2023, notes on file with Amnesty International. 


How does the social card registry work?

The Social Card registry consolidates the personal and other data of social assistance applicants and recipients from multiple existing databases. These include databases managed by the Ministry of Interior, Tax Administration, Pension and Insurance Fund, Employment Service, and Geodetic Institute (housing records). The data is consolidated into a central registry that can be accessed by 1,400 social protection sector employees, including in Centres for Social Work, Child Protection Service and Service for Protection of Veterans Affairs. The Social Card registry uses automation to pull data from these databases and for periodic updates. The registry also uses automation to process individuals’ data in order to identify possible data discrepancies, particularly discrepancies that may affect the eligibility criteria for social assistance. 

It is not only individuals who apply or directly receive social assistance who have their data processed under this system. The Social Card Law also allows the registry to pull and process the data of people related to the applicant. This is done according to provisions laid out in the Law on Social Protection, which mandates that data on related persons is needed to assess an applicant’s claim.54 It is not clear whether or how individuals are informed that their data is being processed in relation to another person’s social protection claim. According to Serbia’s domestic data protection legislation, which is harmonized with the EU‘s GDPR, any data use should follow the principle of data minimization, which requires the authorities to collect and process only data needed for a specific purpose.55 However, the Social Card Law allows for processing the data of related persons without clearly specifying what “an influence on the exercise of rights” means. Furthermore, the Social Card Law contains provisions for processing data on an individual‘s “ex-extramarital partner.”56 In theory, this means that the Law could allow the use of personal data about an applicant’s former partner(s) to potentially exclude the applicant from social protection, regardless of whether they are, or could be, benefiting from the former partner(s) income or assets.57 This also means that a person who is neither a beneficiary nor an applicant to social assistance could have their data in the Social Card registry in relation to another person’s claim without their knowledge. 

While the Social Card registry does not itself make automated decisions about who should receive social assistance, it runs periodic automated checks to identify possible discrepancies between data held on an individual or a household that is pulled from various databases into the centralized registry and eligibility criteria for social assistance. This could include records of income or assets that place an individual or household above the eligibility threshold. In cases of discrepancy, the system sends a notification to a social worker in the local Centre for Social Work, who is then charged with investigating the case.58 The notification is assigned a technical gradation that denotes the urgency with which the social worker must investigate. 

The Social Card Law states that the information contained in the Social Card registry is the authoritative data source for the social worker and their decision-making process.59 If an applicant or recipient maintains that some data are incorrect, the onus is on them to correct or update their data in the original database.60 

A technical gradation notification is attached to each case, denoting the urgency with which a social worker should investigate. Red denotes the most urgent cases, yellow is used for the less urgent ones, and green indicates no urgency. Cases flagged as red relate to either reviewing an individual’s continued eligibility for receiving social protection or investigating the possibility of individuals receiving additional forms of social protection.61 This system, therefore, relies on semi-automated decision-making, whereby the registry selects cases for social workers to review and requests them to conduct and complete investigations to determine whether an individual should continue to receive social assistance or is eligible for additional forms of social protection. 

The data of individuals or households that lose social assistance is no longer processed, but it remains stored in the Social Card Registry in a “passive state” for ten years.62 To reapply, people have to wait three months before submitting a new social assistance application. 

The Social Card Law also allows data to be collected and processed for purposes other than determining social protection eligibility. These include the “creation of social policies, through determining the socio-economic status of the individual and related persons and the wider community”.63 The law does not clearly state what these social policies are nor how they are generated, but it clarifies that data used for these purposes would be anonymized.64 According to the Ministry of Labour, the Social Card registry includes a Data Warehouse that processes depersonalized information on social welfare recipients to create various reports. These include statistical breakdowns of specific benefits, numbers of recipients segregated by age and residency, reports on the monthly uptake of individual benefits and other analyses that could be shared with the Serbian Institute for Social Protection.65 The Ministry maintained that such data is depersonalized and that processing it, therefore, cannot lead to human rights or data protection violations, but they did not provide enough information, such as the types of reports created or show examples of what these reports look like for this claim to be confirmed or refuted. There is also a risk that such reports could potentially be used to identify patterns and trends in data, such as fraud, which could inform targeted policies that could have a disproportionate impact on certain communities. 

In their written responses, the Ministry of Labour rejected the characterization of the Social Card registry as a system that included semi-automated decision-making or any level of automation and maintained that social workers had full autonomy to make decisions on applicants’ eligibility.66  This betrays a fundamental lack of understanding of what automation entails. There is no doubt that the Social Card registry uses automation to pull data from other databases into the registry. The Social Card registry also uses semi-automated decision-making by using automated data processing to identify cases for human review, assigning these cases a technical gradation, and providing social workers with the data needed for their review.67 

A review of the Social Card Law which creates the Social Card registry, World Bank policy documents on the operation of social registries globally, tender documentation for the design of the Social Card registry and correspondence with the Ministry of Labour, along with documents provided by recipients who lost social assistance and interviews with social workers, recipients and partners corroborate these findings. 

Footnotes

[54] That is “persons who have a closer or further kinship, i.e., property relationship, with the individual and that is of influence on the exercise of rights.” Republic of Serbia, Social Card Law.

[55] Republic of Serbia, Data Protection Law, published in the Official Gazette of the Republic of Serbia, no. 87/2018

[56] Article 6, Para. 4 of the Social Card Law elaborates on the concept of “related persons”

[57] Article 9 of the Social Card Law allows for the processing and collection of “[c]ommon and individual data of persons related to the individual”. Articles 9(4), 9(5), and 9(6) also allow for data collection on a related person’s ability to earn a living, their income, as well as other assets they may possess

[58] “If during the data processing a discrepancy of data on the beneficiary, i.e. related person is determined, a notification shall be prepared and forwarded to the records in the field of social protection[…].” Social Card Law.

[59] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[60] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[61] According to Serbia’s Ministry of Labour there were 106,838 cases during the first year of the application of the Social Card Law of people being told that they could exercise an additional right to social assistance. The Ministry, however, did not provide figures on how many people were flagged as not being eligible for social assistance and subsequently removed from the system. Official correspondence from the Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International. 

[62] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International.

[63] Article 4(5) of the Social Card Law allows for the Ministry to conduct “statistical, socio-economic and other research, data analyses and preparation of reports necessary for the performance of tasks within the competence of the ministry responsible for social issues and veteran and disability protection”.

[64] Article 16(3) of the Social Card Law.

[65] Meeting with representatives of Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 28 April 2023, notes on file with Amnesty International.

[66] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International.

[67]  Semi-automated decision-making refers to an algorithmic decision-making system where a human is involved in the decision-making process. These systems are often used to select cases for human review or to assist humans in the decision-making process by providing information and/or suggested outcomes. In the context of the Social Card registry, here are specific examples of the use of automation: automation is used to aggregate data into the registry from the databases held by other government agencies and also to periodically update it; automation is used in data processing; if during the automated data processing there is a discrepancy of data on the beneficiary, a notification is prepared and sent to the Centre for social work for a social worker to investigate; notifications are assigned technical gradations which denote the urgency with which a social worker needs to investigate a claim.


The impact of the social card registry

The introduction of the Social Card registry has, to some extent, reduced the administrative burden for some existing recipients. Automatically pulling available data from pre-existing databases has shortened a lengthy application process for people who previously had to visit numerous offices to collect paper copies of documents to prove their eligibility for social assistance.68 The registry has also allowed for the quick determination of whether existing recipients are eligible for additional assistance. According to the Ministry of Labour, in over 100,000 cases, existing recipients were notified that they qualified for additional benefits, such as child benefits or free childcare assistance.69 

However, a closer analysis of the implementation of the Social Card Law a year after it was introduced indicates that the Social Card registry has been far less successful in providing an adequate buffer against economic hardship for people most in need. During 18 months of implementation, thousands of people, including recipients living in the most economically precarious conditions, have lost access to social security assistance or had their benefits reduced. Indeed, even the Ministry of Labour’s own figures show a continuously decreasing trend in the number of recipients: as of August 2023, there were 176,000 recipients in the system, which is 35,000 fewer than in March 2022 when the Law entered into force. This figure does not necessarily capture all the people who lost social assistance solely due to the Social Card registry implementation. Without disaggregated figures, which the Ministry of Labour should possess but has not released, it is difficult to determine exactly how many people lost social assistance at any one time due to system discrepancies. 

The law has had a particularly negative effect on Roma communities and people with disabilities, who are disproportionally represented in Serbia’s social assistance system. As Amnesty International’s research below shows, the Social Card registry was imposed on an already flawed and thoroughly inadequate social security landscape, plagued by multiple and intersectional forms of discrimination. Introducing the Social Card registry brought with it all the risks associated with the use of automation in delivering social assistance and reduced social workers’ role, who currently seem unequipped – or not empowered – to address errors of the semi-automated decision-making.  

Footnotes

[68]  This does not seem to be the case universally across all Centres for Social Work. According to local partner organizations who work with applicants directly, some Centres continue to require new applicants and existing beneficiaries to collect paper documentation. Email exchange with A11- Initiative for Economic, Social and Cultural Rights, 17 November 2023.

[69] Ministarstvo za rad, zapošljavanje, boračka i socijalna pitanja, “15 meseci primene registra ‘Socijalna karta’”, 27 July 2023 and Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.  

an illustration of a large group of people looking up a a screen in despair as they read the names of unqualified applicants

What is social protection?

A robust social protection system is necessary to protect individuals and communities against income insecurity. This can occur throughout one’s life, including during periods of unemployment, sickness, parental leave, old age, and due to unexpected economic shocks. Social protection measures that comply with the right to social security are crucial to ensuring that all people, in particular those who are marginalized, or at risk of or already living in poverty, are able to realize their right to an adequate standard of living and other related human rights.125 A robust social protection system should also provide people with a meaningful opportunity to escape poverty. 

In general, social protection refers to a broader range of contributory programmes (financed through contributions made by an individual or on their behalf) and non-contributory ones (funded through national tax systems). Social protection programmes can include: (i) social insurance, such as pensions insurance; (ii) employment and labour programmes, including skills training, unemployment benefits, and job search assistance; and (iii) social assistance and cash benefits for people living in poverty. 

Serbia’s social protection system is comprehensive and complex and makes up a substantial part of the country’s expenditure. Serbia spends over 14% of its gross domestic product (GDP) on social protection programmes, exceeding most countries in the region and even some EU member states. However, public social protection expenditure makes up less than 6% of GDP, with the remaining 9% coming from employees’ and employers’ social insurance contributions. Even before the Social Card registry was introduced, and despite high levels of poverty, there were serious gaps in the coverage and effectiveness of means-tested benefits for people living in extreme poverty. As this research shows, the combination of inadequate benefit levels, low social assistance coverage and overly strict eligibility requirements is excluding many people in vulnerable situations from essential income support. 

In Serbia, social protection consists of social assistance, social insurance and employment and labour programmes. 

Social assistance consists of: 

  • means-tested benefits that specifically target people living in poverty, including financial social assistance, child allowance, one-off assistance, and energy subsidies; and  
  • categorical benefits that target a whole group regardless of need and include parental allowance, maternity allowance, and war-veteran benefits. These are considered non-contributory benefits.  

Social insurance programmes include pension insurance that recipients pay towards throughout their working life. This is a contributory benefit. 

Employment and labour programmes consist of unemployment benefits, skills development training, self-employment support, job-search assistance, and other programmes. 

Cities and municipalities can also provide additional financial and in-kind support to people living in poverty. This includes a range of financial and other benefits, including access to soup kitchens, one-off financial assistance, and energy and transport subsidies. Soup kitchens, which are managed by local municipalities and the Red Cross, provide over 30,000 people with at least one daily cooked meal in over 70 municipalities across Serbia.70 

Footnotes

[70] Serbia’s Social Protection law allows local municipalities to provide additional means of material support, including access to soup kitchen or subsidies for local transport, to the people residing on their territories. Social Protection law, art. 111. Due to multiple providers on municipal level it is difficult to determine how many people use this service and figures in the text reflect the provisions made by the Red Cross. 013 Info, Svaki treći korisnik narodne kuhinje je dete, da I ova deca ne budu zaboravljena, 4 April 2022, https://013info.rs/srbija/svaki-treci-korisnik-narodne-kuhinje-je-dete-da-i-ova-deca-ne-budu-zaboravljena/ 


The prevalance of poverty

Successive economic shocks, most recently caused by the Covid-19 pandemic and the war in Ukraine, have seriously impacted living standards in Serbia and disproportionally affected people already living in poverty. Serbia has experienced steadily rising inflation, which reached 15% in December 2022, and a soaring cost of living. While the inflation rate gradually stabilized in late 2023, it remained higher than in other European countries, with the prices of essential food items, electricity and transport also staying considerably high

Serbia has simultaneously suffered from persistently high poverty rates. The country’s absolute poverty rate stands at almost 7%, meaning that almost half a million people are unable to meet their basic needs. The share of children under 18 living in absolute poverty is higher than the national average and exceeds 8%, leaving children overrepresented among people living in poverty. Despite some notable improvements in the country’s overall economic conditions, the number of people living in absolute poverty has hardly changed since 2015

According to the government’s data from 2022, over 21% of people in Serbia were living in poverty,71 while 28% were at risk of poverty and social exclusion. Children, people living in rural areas and minorities, such as Roma, were most affected. Recent research by United Nations (UN) agencies in Serbia showed that households with children were among the most vulnerable to experiencing poverty, with poverty rates for children aged under five reaching over 17% in 2022.72 According to the same study, over 21% of children living in larger households of more than four members lived in absolute poverty, while the figure increased to 43% for children living in rural areas. In terms of socio-economic indicators, Serbia falls well below EU averages.73 

Members of marginalized communities, such as Roma, people with disabilities and women, are particularly vulnerable to poverty. Surveys show that 83% of Roma live in conditions of pronounced material deprivation, meaning that they are unable to provide for their basic needs, such as food. A 2019 government survey further indicated that 32% of substandard Roma settlements were not connected to electricity, and 38% did not have access to the water supply grid.74 In addition, Roma  – perhaps more than any other group in Serbia – face persistent discrimination in all spheres of life, including significant barriers to accessing health, education, employment and government services. People with disabilities in Serbia are also at higher risk of experiencing poverty, partly due to significant barriers to accessing employment or education, key factors that increase a person’s risk of living below the poverty line.75 

Women in Serbia have the highest unemployment rate in the Western Balkans (this refers to Albania, Bosnia and Herzegovina, Montenegro, Kosovo and North Macedonia), partly due to family care responsibilities. Women are also most likely not to have pension insurance. Across nearly all ages and demographic groups, women tend to be at a very high risk of poverty. Crucially, many individuals may be experiencing intersecting forms of discrimination that together increase their risk of living in poverty. 

Serbia has a higher income distribution inequality than any other country in Europe. Its Gini coefficient – or a measure of the distribution of income across population measuring inequality – is 35, which is significantly higher than the EU average of 29.76 A large share of people live in very low work-intensity households, namely, households whose working-age members work less than 20% of their full potential, while a high proportion of people who do work are often in informal, part-time, or temporary employment. The EU statistics on income and living conditions (SILC) survey showed that the population’s living standard based on income is falling, with the living standard of the poorest falling at an even higher rate. 

Footnotes

[71] The relative poverty line in Serbia is 24,064 Serbian dinars (212 euros) a month.

[72]  See https://www.unhcr.org/rs/en/21837-vulnerable-households-face-unique-challenges.html and https://www.unicef.org/serbia/sites/unicef.org.serbia/files/2023-01/Child%20Poverty%20Projections.pdf

[73] See EU data at https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20230614-1. Also, see EU Eurostat data on statistics and living conditions in enlargement countries at https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Enlargement_countries_-_statistics_on_living_conditions 

[74] Official Gazette of the Republic of Serbia, “Strategy for the Roma inclusion in the Republic of Serbia for the period 2022-2030”, No. 23/2022, 17 February 2022, p. 50.

[75] Beker, K., Janjic, B., “Covid-19 measures and social and economic rights of women with disabilities in Serbia”, 2021, Out of Circle Belgrade & FemPlatz; Perisic, Natalija and Vidojević, Jelena, “Divided by poverty and social exclusion-Roma and persons with disabilities in Serbia”, 2015, pp. 142-159.

[76] A household with very low work intensity can be defined as “the number of persons living in a household where the members of working age worked a working time equal or less than 20% of their total work-time potential during the previous year”. Eurostat Statistics Explained, Glossary: Persons living in households with low work intensity, https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Glossary:Persons_living_in_households_with_low_work_intensity 


Low public expenditure on social assistance programme

Although the Serbian government’s total expenditure on social protection is relatively high compared to other countries in the region and almost at the same level as EU countries, its budget is largely dominated by pensions, which constitute over 70% of the total social protection spending, while funds for non-contributory social assistance remain very low. Only 5% of Serbia’s social protection expenditure goes to means-tested, non-contributory programmes, which is well below the EU average of 12%. Despite growing inflation and the persistently high number of people living in absolute poverty, government expenditure on non-contributory benefits fell from 3.1% of GDP in 2019 to 2.7% of GDP in 2021 and stayed at that level in 2022. This reduction in funding had a negative impact on financial social assistance and child allowance, which both target those living in extreme poverty. Serbia’s spending on means-tested benefits, which was below 1% of GDP in 2019, was well below the EU average of over 3% of GDP. The Regional Cooperation Council’s 2021 review of Serbia indicated that only 0.3% of GDP went to social assistance, which was below even typically low regional standards. 

The World Bank noted that this reduction in financial social assistance was partly driven by the long-term trend of increasing expenditures on categorical programmes, such as pro-natal benefits or war veteran benefits. It warned that without a major policy change, this trend may ultimately threaten the sustainability of financing social assistance for those living in extreme poverty.156 CESCR also expressed concern about the “persistently low level of funding allocations” undermining people’s realization of their economic, social and cultural rights. It urged Serbia to dedicate sufficient human and financial resources to reducing poverty and ensuring that their measures reach those in need.77  

Footnotes

[77]  UN CESCR, Concluding observations on the third periodic review report of Serbia, 6 April 2022, UN Doc. E/c.12/SRB/CO/3, paras. 24, 25, and 51.


Restrictive eligibility criteria for social assistance programmes

Financial social assistance is the only explicitly means-tested anti-poverty programme in Serbia. According to Serbia’s Law on Social Protection, only people with an income from employment, assets or other sources that fall below the monthly financial social assistance are eligible for financial social assistance.78 As of October 2023, this amount is set at 11,445 Serbian dinars or 97 euros per person. In addition, in order to qualify, individuals or families cannot own a home that exceeds one room per person, land over 0.5 hectares, or have movable assets with a value exceeding six times the basic social assistance.79 People who earn more than the monthly social assistance or own a home different to that deemed suitable under the law, are not entitled to financial social assistance unless the property is mortgaged. Eligibility criteria for households and families are also related to the unemployment status of able-bodied members. 

The eligibility threshold, that is, the maximum income and benefit levels, are updated twice a year in April and October, to take into account consumer price changes. 

There is a broad recognition, however, that Serbia’s eligibility criteria are too restrictive to effectively protect people against poverty. The maximum income threshold to qualify, currently set at 11,445 Serbian dinars (97 euros), is so low that it falls below the absolute poverty line of 12,500 Serbian dinars (106 euros) per month. This excludes many people, who earn slightly above the threshold but still live in poverty, from social assistance, leaving them without any form of support. 

Some eligibility criteria have also had a disproportionate and discriminatory impact on specific populations. Rigid asset testing, such as for land ownership, without ascertaining land use or land productivity, is hugely disadvantageous to people living in poverty in rural areas. Many people live in dwellings that might appear large on paper but are often profoundly inadequate for a dignified or healthy life. They may also have inherited land without any real value because it is not agriculturally productive or is shared among multiple siblings and, therefore, is difficult to sell. This means a substantial number of low-income agricultural families, who are already disproportionately affected by poverty, fall through the social assistance safety net. 

There is a benefit cut-off for a fifth and any further children and a requirement that children receiving social assistance must “live, go to school and regularly attend class in the territory of the Republic of Serbia,”80 both of which disproportionately affect Roma families. Many Roma families have more than four children and face extreme poverty, social exclusion and discrimination that limit their access to education.81 Such restrictions are also highly unusual for CoE member countries, where social support payments for families are typically scaled by size and do not include an upper limit on the number of children receiving benefits. The policy on four children excludes a small percentage of families in Serbia from social assistance, however, these constitute some of the families most in need. Not only does the social assistance child cap  deprives children in these families of the right to social security and traps them in poverty; it also discriminates against Roma, who are disproportionately impacted by this rule, as well as other families with more than four children. In 2022, the UN Committee on Economic Social and Cultural Rights (CESCR) urged Serbia to review its eligibility criteria and remove discriminatory provisions to ensure greater uptake of social assistance benefits.82

Footnotes

[78]  Law on Social Protection, Official Gazette of Republic of Serbia number 24/2011 and 117/2022, Article 81, para 1.

[79]  Law on Social Protection, Official Gazette of Republic of Serbia number 24/2011, Articles 82 and 83 and number 117/2022, Article 81, para 1.

[80] Republic of Serbia, Law on Financial Support for Families with Children. 

[81] The illiteracy rate among Roma people is 10%, far above Serbia’s national average of 2%. According to government data, one-third of Roma have not completed primary education, 12% have completed secondary education, whereas only about 1% of Roma have completed two years of post-secondary or higher education. Eighty per cent of women in Roma settlements are literate. See Strategy of Social Inclusion of Roma for the Period from 2016 to 2025 – Serbia, 2 Sep 2016, https://www.rcc.int/romaintegration2020/files/admin/docs/25271eee1fb46a73d48630d6d4d63bec.pdf, p. 36. 

[82] UN CESCR, Concluding observations on the third periodic report on Serbia, 6 April 2022, UN Doc. E/c.12/SRB/CO/3, paras. 50 and 51 (b).


Poor reach of social assistance

While financial social assistance remains essential, it continues to be inadequate and insufficient for covering the basic needs of those who most need financial support. Many people living in poverty are unable to obtain social assistance due to overly strict requirements, as explained above. Those who do qualify are still unable to meet their basic needs even after receiving financial support. The World Bank estimates that social assistance in Serbia reaches less than 10% of the people living in poverty.83 This figure is significantly lower than for social assistance coverage in the new EU Member States, including Slovakia, Hungary and Romania, where social assistance reaches 25% of those living in poverty.83 Moreover, over half of children from poor families fall outside the social assistance net despite the fact that children are the most vulnerable to poverty. 

Social assistance covers about 3.5% of Serbia’s population or only 40% of people who live in absolute poverty. The standard of living for this demographic has not only failed to improve over recent years but continues instead on a rapidly downward trend

At the same time, the number of social assistance recipients has seen a steady decrease over the past years, dropping from 268,000 people in 2018 to around 210,000 in 202084 and even lower to 177,000 in 2023.84 This decreasing number of recipients does not indicate a significantly improved standard of living, as the Serbian authorities argue85 because the percentage of people living in absolute poverty has remained stubbornly high at 7%. In its annual assessment of Serbia, the European Commission noted that “means-tested social assistance programmes, with strict eligibility criteria, do not sufficiently reach the poor.” It also urged Serbia to improve the quality and reach of its social services.       

Footnotes

[83] Social assistance reaches only 3.5% of Serbia’s population even though 7% of the population lives in absolute poverty, and over half of all children from poor households fall outside of the coverage. See World Bank, Social Protection Situational Analysis: Serbia, 1 January 2022, https://elibrary.worldbank.org/doi/abs/10.1596/37870, p. 10.

[84] European Centre for Social Welfare Policy and Research, Performance of Western Balkan economies regarding the European Pillar of Social Rigths, 2021 Review on Serbia, 2021, p. 41

[85] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[86] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.


Inadequate level of social assistance

The level of financial assistance available for families in Serbia continues to be inadequate. It is failing to pull people out of poverty or even alleviate their economic hardship. As outlined above, in 2023, the country’s monthly social assistance was 11,445 Serbian dinars (approximately 97 euros) per person. This equals to around 20,000 Serbian dinars (170 euros) for a family of three, including one child. This amounts to less than half the cost of the minimal consumer basket (443 euros) and falls well below the relative poverty threshold of 43,315 Serbian dinars (270 euros). The inadequacy of the social assistance is even more obvious when compared to the monthly minimum wage, fixed at 53,000 Serbian dinars (460 euros) in 2023. The average social assistance is also below the absolute poverty line of 12,500 Serbian dinars (106 euros), which is far too low to meaningfully address poverty. 

In its 2021 Conclusions on Serbia, the European Committee of Social Rights (ECSR) reiterated that the level of social assistance paid to persons without resources continued to be well below the absolute poverty threshold. The UN Committee on Economic, Social and Cultural Rights (CESCR) and the EU have both repeatedly urged Serbia to increase its social security benefits to ensure that recipients can enjoy an adequate standard of living.87 

icon of a person

Siniša, a man aged 46 from Novi Sad, lives alone in a 12 square metre room with a leaking ceiling. He does not have a functional kitchen and shares a toilet with a dozen other residents in the same building. Siniša spent his childhood in multiple foster homes and orphanages and carries deep trauma from witnessing abuse and living without adequate care throughout his life.  

Siniša told Amnesty International that he is not able to work due to various physical ailments, such as persistent stomach problems and mental health difficulties. Yet, according to Serbia’s Law on Social Assistance, he is considered to be “able-bodied” [see below] and, therefore, not eligible for a full 12 months of social assistance. Siniša’s only steady monthly income is the 5,800 Serbian dinars (50 euros) that he receives from social assistance for nine months a year. “Of course, I can’t survive on the assistance,” he said. “The assistance goes straight on paying my monthly rent, which is 40 euros (around 5,000 Serbian dinars). I also have to pay electricity and communal tax, which is an additional 5,000 Serbian dinars per month. And what about food?!”  

To survive, Siniša has to rely on the kindness of his neighbours, for whom he does small chores, and humanitarian organizations that sometimes bring him care packages. Instead of providing a buffer against poverty and helping people experiencing a financial crisis the social assistance programme only traps people, like Siniša, into a permanent cycle of poverty. 

Footnotes

[87] UN CESCR, Concluding observations on the third periodic report on Serbia, 6 April 2022, UN Doc. E/c.12/SRB/CO/3 also European Committee of Social Rights, Conclusions 2017 – Serbia.


Social assistance system is penalising for trying to make ends meet

While failing to effectively shield people from poverty, Serbia’s social assistance system also penalizes them for being employed and doing temporary work. People receiving social assistance are not allowed to combine it with having income from formal or informal employment. Earning any income risks people having their benefits reduced or completely removed if the income exceeds the maximum threshold for social assistance. People interviewed by Amnesty International said that this discouraged those wishing to proactively search for work, creating social welfare dependency and ultimately preventing people from escaping poverty long-term. Dinko from Belgrade told Amnesty International: “If I could work and get the assistance, we would live an OK life.” For him, “all people who can [work] would like to work, but they are scared they will be kicked out of social assistance”.88 

Data on the effectiveness of social assistance in Serbia show that it has a negligible impact on reducing poverty. Financial assistance is the least effective for households with children and does not lift people out of poverty. Almost 60% of young recipients of social assistance (aged between 15 and 29) come from families that also receive social assistance, indicating that inherited, long-term poverty is a norm in Serbia.89 

This is particularly the case for marginalized groups such as Roma, who already have extremely limited access to the labour market, education and employment due to structural discrimination. For many, low paid informal work is often their only realistic chance of generating an income. They, therefore, find themselves in a situation where their income is not enough to sustain a minimum standard of living and have to rely on social assistance, which is not sufficient to make ends meet. This is compounded by the eligibility threshold for assistance being so low that trying to find informal work to supplement social assistance and cover basic family needs can mean risking losing social assistance. The existing system penalizes people harshly simply for trying to make ends meet instead of providing meaningful opportunities for getting out of poverty – a key function of any effective social protection system. 

Typically, countries will exclude small, irregular or informal incomes from the calculation of a household income that is considered to determine eligibility for benefits. Counting irregular income does not improve efforts to reach people in need of assistance, and it also risks categorizing informal income as fraud and discouraging work,90 something that most people interviewed by Amnesty International in Serbia also stressed. Moreover, as testimonies in this report widely confirm, this practice forces families living in poverty to make unacceptable trade-offs – having to either continue receiving social assistance alone or earning a small income from informal work – neither of which on their own is sufficient to cover basic needs.

Footnotes

[88] Amnesty International interview with Dinko, 25 April, 2023, Belgrade, Serbia.

[89] European Commission, European Social Policy Network (ESPN) Thematic Report on Social Investment in Serbia, 2015, p. 15

[90] Grosh, Margaret; Leite, Phillippe; Wai-Poi, Matthew; Tesliuc, Emil, “Revisiting Targeting in Social Assistance: A New Look at Old Dilemmas. Human Development Perspectives”, 2022, p. 357.


Narrow definition of working capacity under the law

The Law on Social Protection sets an unusually high threshold for people who are considered unable to work, using a purely medical assessment of disability. This means that people who do not fit into the medical categories used, including those with mental health illnesses and some physical ailments, are frequently classified as “able-bodied” and denied social assistance for the full amount or for all 12 months. Only those who are “completely unable to work” or who have “complete” and “multiple” difficulties, making them hard to employ under general or special conditions, can receive social assistance for the full year.91 The UN Committee on the Rights of Persons with Disabilities criticized Serbia for the continuing use of the medical model of “incapacity”, which was disadvantageous for persons with disabilities and not in line with human rights standards.92 

In cases where people are classified as “able-bodied”, or in households where more members are classified as “able-bodied” than not, individuals only receive nine months of social assistance.93 In theory, this gives them the possibility of getting temporary or seasonal employment in the three remaining months. In reality, however, this leaves people who, for all practical purposes, are not able to work or gain temporary employment, without any assistance for at least three months each year, thereby exacerbating their economic hardship. As this report confirms, this gap in assistance pushes people into a desperate situation where they have no alternative but to survive by begging for food, collecting and selling secondary raw materials such as plastic and glass bottles, cardboard and scrap metal in landfills, or borrowing money from friends and family and getting into debt that they have no means of repaying. 

For some, like Siniša, the disruption in benefits happens at the most financially difficult time of the year, from October to January. This leaves him without assistance during the four coldest months when bills for electricity and gas are also significantly higher. “It is especially tough during the gap in social assistance. By the time the Centre for Social Work reactivates the benefits, the three months usually turn into four and a half months, during which I have absolutely no help. It’s not easy. I barely ever turn on the heating, even when it’s cold,” Siniša said.94  

According to a Serbian non-governmental organization (NGO), the A11 – Initiative for Economic and Social Rights, the gap in assistance pushes people further into poverty. “Some utility companies charge interest or penalties on unpaid bills while others, like electricity providers, cut their services for those who fail to pay the bills on time. This creates an insurmountable problem for poor people who are left without assistance for several months.”95 

Footnotes

[91] Law on Social Protection, Article 85, para. 2, items 4 and 7.

[92] UN Committee on the Rights of Persons with Disabilities, Concluding observations on the initial report of Serbia, 23 May 2016, CRPD/C/SRB/CO1, paras 53 and 54.

[93] Law on Social Protection, Article 85, para. 3.

[94] Amnesty International interview with Siniša, 27 September 2023, Novi Sad, Serbia. 

[95]  Interview with a representative of A11 – Initiative for Economic and Social Rights, 28 September 2023, Belgrade.

an illustration of a family peaking their heads through a giant pile of news papers and bill notices.

Chapter 5 – Flaws in the social card registry further restricting the right to social security  

Serbia’s Social Card Law and the creation of the Social Card registry were meant to address some of the gaps in the country’s social security system, enable a fairer distribution of benefits to the most economically vulnerable groups and improve control over the delivery of social assistance.184 An integrated system that pulls data from different institutions was designed to build a comprehensive socio-economic profile of recipients to determine their eligibility for social assistance. The Social Card Law does not prescribe eligibility criteria for different types of assistance (they are determined by the Law on Social Protection and other applicable laws), but instead, the authorities argue, it provides a legal framework for creating a technical infrastructure to determine applicants’ socio-economic status. 

However, Amnesty International’s findings indicate that the Social Card registry is more than merely a technical infrastructure housing relevant data. Instead, it appears to operationalize an already restrictive and inadequate set of eligibility requirements and risks exacerbating existing marginalization and exclusions in the system. Using a data-driven semi-automated decision-making approach also introduces new human rights risks into an already flawed social assistance system, such as poor data quality, automation bias, and potentially increased surveillance of recipients. Introducing this kind of automation into an already restrictive social security framework without addressing strict eligibility requirements, ensuring full accuracy of entry-level data, and putting in place meaningful safeguards against these risks was bound to further undermine people’s right to social security. 

The Social Card registry has placed a premium on data in the registry and reduced social workers’ ability to assess data accuracy or collect and add information into the registry that could more accurately capture marginalization and social deprivation or provide context, such as living conditions or precarious housing. It has also put the onus on individuals to disprove data errors and collect evidence to support their eligibility claim when the system detects a discrepancy in their data, rather than requiring social workers to conduct the investigation. 

Amnesty International found that the use of automation in the Social Card registry had a number of flaws and added more complexity and arbitrariness to the system of assessing eligibility for social assistance. These flaws are discussed in more detail in the sections below and can be broadly categorized into five groups: 

  1. significant potential for errors and incorrect/incorrectly classified data included in the Social Card registry; 
  1. problems with data integrity and the use of unrepresentative and unreliable data on income, assets or housing that, when presented without any nuance, is not an adequate measure of people’s often complex economic status and does not reflect an individual’s economic hardship; 
  1. the system design is set up to prioritize tracking household changes that lead to the loss of assistance rather than facilitate access to benefits; 
  1. Social Card registry operationalizing existing restrictions to social assistance and exacerbating exclusion; 
  1. social workers lack autonomy to make decisions on eligibility. 

The section below also highlights the often profoundly negative impact of the Social Card registry on people’s already strained access to social assistance.


The problem of incorrect or incorrectly classified data

Because the Social Card registry pulls data from a wide range of databases to help assess applicants’ eligibility for social assistance, the accuracy of the data in the originating databases plays a huge role in ensuring fair application outcomes and continued receipt of social assistance. Indeed, the utility of a dynamic registry is based on the premise that each individual database that is a part of the network meets a certain quality threshold and provides quality data. Research for this report found that the source data collected by the Social Card registry has a significant potential for being inaccurate. In cases documented for this report, people lost social assistance because the data collected by the registry was either completely incorrect or was classified incorrectly as income, even though it was not. 

Adra and Brane are a young couple with four children who live in Belgrade. They lost social assistance in April 2023 because the Social Card registry showed that Brane had 100,000 Serbian dinars (850 euros) deposited in his bank account. Brane was in the process of challenging this decision when Amnesty International interviewed him. He insisted he had no money or an active bank account: “I was surprised. 100,000 dinars!? I have never even seen that much money. The only money we get is the social assistance we receive every month. I don’t even have a bank account!” A few weeks later, he managed to obtain information from the Tax Administration Office proving that the Office had made a mistake.96 

Bogdan, also from Belgrade, had a similar experience. When he tried to renew the assistance he and his family were receiving in March 2023, he was told that he had “too much money” in his bank account. “This was ridiculous. I don’t even have a bank account, and 90,000 Serbian dinars (770 euros) is a huge amount of money,” he said. “I tried to get information from the Centre for Social Work about this bank, but they were not able to tell me. They said it was my job to get the paperwork confirming that this was not correct.” At the time Amnesty International spoke with him, Bogdan was, with the help of a local aid organization, trying to get a document from the Tax Administration proving that he had no bank account nor the stated funds to his name so that he could reapply for social assistance.97 

Jelena from Belgrade is a single mother with a young son. She worked only two days in August and September, respectively, earning her a total of 6,000 Serbian dinars (51 euros). However, the Social Card registry received information from the Tax Administration that she had earned 150,000 Serbian dinars (1,200 euros) in August, September and October. Despite Jelena submitting evidence to the Centre for Social Work proving the exact amount of time she had worked and how much she had earned, her social assistance was removed. The proof she submitted included a decision by the Central Register of Compulsory Social Insurance, the state body that keeps records of all earnings based on which compulsory social insurance contributions are paid. This showed that in August, September and October, Jelena indeed worked for only four days and earned 6,000 Serbian dinars (51 euros). According to Jelena, the Centre for Social Work staff told her that they “can’t do anything, it’s the computer that says it”.  

Mirjana’s case is perhaps the most emblematic of the limitations and inadequacies of using only reductive data to determine someone’s social assistance eligibility. In February 2023, Mirjana’s daughter died unexpectedly. Living on modest social assistance and in a social housing complex, Mirjana was not able to cover the cost of her daughter’s funeral. A local human rights organization stepped in to assist with the funeral arrangements and deposited 20,000 Serbian dinars (around 170 euros) into Mirjana’s bank account in the form of a donation.98 This money was instantly flagged by the Social Card registry as “income”, and the local Social Work Centre then informed Mirjana that she no longer qualified for social assistance.99

To complicate matters, Mirjana and other residents in the same social housing complex were indirectly receiving compensation from a local company whose development project had caused them to be evicted from their original homes in the Vinca settlement. The company paid the cost of their new social housing directly to the company managing the housing complex, with residents not receiving any funds directly. However, because they were named as beneficiaries in the transfer, their names were flagged by the Social Card registry, and Mirjana’s Tax Administration record, therefore, showed additional income “to her name”. 

Mirjana appealed the decision to remove her social assistance, arguing that the Social Card registry or its originating databases had incorrectly classified these donations as income. Within two months of a profound personal tragedy, Mirjana found herself in a desperate situation, without social assistance – her only means of sustenance. She also faced a prolonged and uncertain bureaucratic battle to reinstate her lost benefits due to inaccurate information about her financial situation.100 In November 2023, Mirjana was told that her second appeal was successful.101 

In meetings and correspondence with Amnesty International, Serbian authorities maintained that the Social Card registry itself could not include inaccurate data. However, they agreed that there is potential for source-level data pulled from various databases into the Social Card registry to contain errors,102 in which case recipients are themselves responsible for correcting them directly in these databases. While the Ministry of Labour noted that it had not documented any mistakes in source-level data since the system was introduced and that, for example, the data on income was found by them to be “100% accurate”,103  the above testimonies poignantly contradict this. An absolute certainty about the accuracy of the new and not yet fully tested system professed by the authorities could undermine any meaningful effort to identify gaps and invest in appeals and other grievance mechanisms to allow recipients to challenge errors. 

Although the Ministry of Labour or the local Centres for Social Work might not be able to correct errors in databases managed by other government institutions, Serbia’s government as a whole is responsible for ensuring that all participating citizens’ databases that provide key information and source data to the Social Card registry on applicants and recipients, are up to standard and contain accurate information. A dynamic, that is, regularly updated and agile, social registry cannot be effective or fair if it relies on less dynamic and potentially unreliable databases feeding it with information and data. More importantly, as cases documented here show, such errors can have devastating consequences for people’s access to social assistance. 

Footnotes

[96] Amnesty International interview with Brane, 25 April 2023, Belgrade, Serbia. 

[97] Amnesty International interview with Bogdan 25 April 2023, Belgrade, Serbia.

[98] Based on the review of complete documentation provided by “Mirjana” and A11- Initiative for Economic and Social Rights, including the Notification on submitted individual tax application (so called “PPP PD application”) submitted along the relief payment. On file with Amnesty International.

[99] Belgrade Centre for Social Work, Decision on cessation of social assistance. On file with Amnesty International.

[100] Amnesty International interview with A11 Initiative, 28 September 2023, Belgrade, Serbia. 

[101] In official response to the “right of reply” letter from Amnesty International ahead of the publication, Ministry of Labour disputed most of the facts of the case and argued that “Mirjana” lost social assistance due to “a social worker error”, rather than Social Card registry or Tax Administration error, and insisted that any errors in data were not possible. Amnesty International had access to “Mirjana’s” complete file and stands by the testimony as presented.

[102] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[103] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.


Problems with data integrity

The Social Card registry’s heavy reliance on raw data, especially for individuals who are marginalized, potentially increases the risks of data integrity. The authorities in Serbia have not conducted a comprehensive outreach programme to marginalized communities who received social assistance prior to the rollout of the Social Card registry to inform them about eligibility conditions and procedures, as well as the increased importance of good record keeping. In a significant number of cases documented by Amnesty International, recipients were not aware that they needed to update their data on ownership of assets in the original databases and that their failure to do so would have resulted in the loss of social assistance. 

Among other conditions, the Social Card registry uses data on assets purportedly owned by individuals and households to assess their eligibility for social assistance. Those found to have assets above the threshold prescribed by the Law on Social Protection are flagged for investigation by the Centres for Social Work, often leading to the loss of social assistance. However, as testimonies below show, the data on assets alone pulled from other databases did not always accurately reflect people’s true socio-economic conditions, nor did they take into account the systemic challenges to good record-keeping facing some marginalized communities.  

One social worker told Amnesty International about a case where an individual was flagged for living alone in a house that had three rooms and, therefore, found ineligible for social assistance. However, in reality, only one of the rooms was fit for human habitation since the other two rooms did not have a roof. “The raw data clearly has its limitations,” she said.104 In her mind, it was essential that the social worker verifies the data from the Social Card registry through a field visit. 

Brane is a young man from Belgrade who lives with his wife and four small children, including a son with severe epilepsy. Like others Amnesty International spoke to, they lost social assistance in part because of a car registered to his name. Brane had bought a used car several years earlier to make it easier to travel frequently to the hospital with his son. However, because Brane failed to pay back the personal loan he took out to buy the car, he can no longer register, use or sell it, and the car now sits unused in front of his house. The Centre for Social Work told Brane that unless he sold it, his family could not get their social assistance back. 

Although Brane filed an appeal, it was rejected. He hoped that he would be able to clear the car problem in the coming months and reapply for assistance after the three-month waiting period required before submitting a new application. Meanwhile, his family survives on child credit and the childcare allowance for their son, who has special needs. “The assistance was everything to us and the children’s life, and that money was spent only on food and children’s education. I am not sure how we will survive from here on,” Brane said.105 

Dinko from Belgrade has a family of nine, including his wife, five children and his elderly parents. He was told by the Centre for Social Work staff in April 2023 that his family could no longer receive social assistance because the Social Card registry records showed that his father owned two cars. Dinko explained that the cars, registered in his father’s name in 2013 and 2016, were never in a driving condition and had been sold for scrap metal years ago. Unfortunately, Dinko’s father never officially de-registered the cars because he was unaware that this was necessary. The cars, therefore, continued to show on his record as assets, jeopardizing his access to benefits.106

Using assets as an eligibility criterion – without proper prior outreach with households and verification by a social worker – completely overlooks the ways in which many Roma and other marginalized people rely on informal work to support themselves. Buying or taking on old defunct cars and selling them for scrap metal is one way to make a modest income. People often sell these cars without formally changing ownership or deregistering them – thereby unwittingly risking their access to benefits. Furthermore, in the cases reviewed for this report, nearly all instances of flagged vehicles referred to cars that were sold years before the Social Card registry was in place.  

A representative of the A11-  Initiative for Economic and Social Rights, a local organization providing free legal aid to welfare recipients, told Amnesty International that many scrap metal vendors fail to transfer titles and register cars in their name after they purchase them from individuals, usually for cash, to avoid paying tax: 

As a result, people are stuck with multiple cars registered in their name even though they no longer own them. The Social Card registry pulls the information from the Ministry of Interior database, and social workers no longer investigate or try to understand why these clearly very poor people have several cars in their name.107

Representative from the A11 Initiative for Economic and Social Rights, 25 April 2023, Belgrade, Serbia

Given the known challenges associated with social registries in other countries, including the fact that social barriers, such as lower literacy levels and existing marginalization, profoundly affect the ability of some communities to access social protection services, robust mitigation measures are vitally important. Such measures include information on how to access benefits, the application process, and the necessary documentation required, but also information on circumstances in which benefits might be suspended or stopped, such as not formally deregistering cars, which is a problem affecting a significant number of recipients. 

Serbia’s failure to take into account the systemic challenges to good record-keeping and access to government offices faced by marginalized communities while designing and implementing the Social Card registry and its failure to conduct effective outreach to affected communities compromised the integrity of data used to determine people’s eligibility and further restricted their access to social assistance. 

Footnotes

[104] Amnesty International interview with a social worker, 27 April 2023, Serbia.

[105] Amnesty International interview with Brane, 25 April 2023, Belgrade, Serbia.

[106] Amnesty International interview with Dinko, 25 April 2023, Belgrade, Serbia. 


System only tracking changes that can lead to the loss of assistance

While the Social Card registry is automatically updated periodically, it appears that it prioritizes tracking positive changes, that is, changes in household conditions which affect eligibility and result in the loss or reduction of social assistance people receive. For example, if the registry determines that a person has earned additional income over the past month, it will immediately flag them for investigation by a social worker, who will then remove them from social assistance or reduce the amount they receive. However, if the same person does not earn the same income in the following month, the registry will not immediately or automatically reinstate the person’s benefits as they are no longer in the system. 

In other words, despite the availability of data on changes to household income levels, for example, the Social Card registry does not equally track the negative trends affecting households, such as loss of income or employment, and automatically triggers the start of benefits or notify individuals that they now meet eligibility conditions. In fact, as the Ministry of Labour explained, the data of people who lost social assistance were no longer processed but remained stored in the registry in a “passive state.”107 The Ministry argued that processing data of people who were removed from the system until they were explicitly asked to do so by the individual would be a data protection violation. However, this does not explain the delay in reactivation of an individual’s file once their income drops below the threshold or why the amount of social assistance received would be fixed based on the earnings of one month and not periodically updated month to month. 

Katarina, a woman aged 40 from Kraljevo, showed Amnesty International documentation from the Centre for Social Work stating that her social assistance from November 2022 to July 2023 was fixed based on the amount of money her household had earned in November 2022 rather than on the household’s actual earnings each month. As Katarina’s household’s earnings came from collecting and selling secondary raw materials, her income varied significantly from month to month. Despite receiving regular updates, the design of the Social Card registry does not seem to allow new information on households to be used automatically to ensure that recipients whose circumstances have worsened from the previous month will receive the full amount of social protection they are entitled to. Instead, they need to wait for three months to reapply.  

Indeed, one of the purposes of a dynamic social registry, such as Serbia’s, is precisely to allow governments to efficiently identify and substitute people who are no longer eligible108 and use real time data to allow people to quickly cycle in and out of the social assistance programme based on changes in their eligibility. Yet, the time it takes to reinstate people’s social assistance, which can take months, contrasts sharply with the speed with which people lose their benefits. It is not clear whether the Social Card registry’s preference for tracking positive trends in household finances that lead to the loss of assistance is driven by policy or a design, but it is a common challenge documented across multiple dynamic registries operating in low and middle-income countries. It is difficult not to interpret this as yet another means of prioritizing potential fraud detection over greater inclusion of households in need. 

Footnotes

[107] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International.

[108] World Bank, Charting a Course Towards Universal Social Protection, p. 90.


Social card registry operationalizing existing restrictions to social assistance

As has already been outlined, Serbia’s social assistance is profoundly inadequate. It is insufficient for covering people’s basic needs, including food, and even less able to realize people’s access to their other economic and social rights, such as education and healthcare. Interviews for this research showed that most recipients spent their social assistance on food, leaving them with no income to buy medication, clothes and children’s school equipment. Yet, the existing social assistance system penalizes people who try to earn additional income to make ends meet. This problem predates the introduction of the Social Card registry. However, the new automated system and its rapid detection of any new circumstances that can result in people’s social assistance being reduced or removed altogether only operationalizes existing restrictive eligibility criteria. This is particularly the case in the context of often insignificant and irregular income earned through informal work that, prior to the Social Card registry, social workers did not take into account when determining eligibility for social assistance. 

Many individuals interviewed for this report lost social assistance after the Social Card registry flagged them for having an income, putting them above the minimum threshold. This income frequently referred to modest amounts made by collecting and selling secondary raw materials, such as plastic and glass bottles, cardboard, and scrap metal. These earnings were often so negligible and irregular that they could not have significantly changed people’s economic situation enough to raise them above the poverty line. Instead, people used this income to supplement their social assistance, which was so low it was insufficient to feed their families. 

Informal work is a necessity for many who live in poverty. This is especially true for marginalized groups who face persistent exclusion from the formal labour market. As noted above, Roma, people with disabilities, and single parents are among the most economically vulnerable groups in Serbia and face multiple barriers to employment. Roma, especially people aged 18-24, are twice as likely to be unemployed than their non-Roma neighbours. A lack of formal education, work experience, training and extreme discrimination are just some of the obstacles disproportionately facing many Roma seeking work. Of Roma men and women who have jobs, almost 70% are informally employed, including collecting recyclables or working in flea markets. Serbia’s most recent Strategy for Roma Inclusion recognized that members of Roma families facing severe deprivation often resort to working in the informal economy by necessity and proposed to gradually formalize this work. 

Some informal Roma settlements almost entirely rely on the collection and sale of plastic, glass and metal to survive and were particularly affected by the introduction of the Social Card registry. In an informal settlement in Kraljevo, over a hundred families lost social assistance in the first months after the Social Card law entered into force because the system detected earnings from the sale of secondary raw materials.109 Although some have had their benefits reinstated since, many faced months without any support, while others are still engaged in lengthy appeals process. 

Edna, a Roma woman aged 33 from Kraljevo, lives with six small children in a 20-person extended household. Her family received regular monthly social assistance of 20,000 dinars (about 170 euros) until May 2023, when the local Centre for Social Work started implementing the Social Card Law: 

One day I received a call from a social worker who informed me that I was no longer eligible for the assistance because my husband made a small income, about 45,000 dinars (about 380 euros) over several months. He was now considered gainfully employed, which makes him ineligible for social assistance. 

Edna, aged 33 from Kraljevo

Edna explained that, like most people in the informal settlement that she lived in, her husband relied on selling scrap metal and glass from landfills to supplement their social assistance because it was not enough to meet their family’s needs. Edna’s home, which Amnesty International visited, was dilapidated and had no furniture. She said they were not able to properly feed their children nor buy them clothes so they could go to school.110

Milan, a man aged 44 from Kraljevo, also lost his social assistance in May 2023, along with many others in his village: 

I lost my social assistance because I made ‘too much money’. This is ridiculous. I literally made 1,000 or 2,000 dinars (8-17 euros) from selling old tins, and that meant I could no longer get monthly financial assistance of 8,000 dinars (80 euros). I am too ill to work full time and the assistance was not enough for a normal life. 

Milan, aged 44 from Kraljevo

Milan, who no longer received social assistance when Amnesty International spoke to him, sold second-hand clothes in the market, which he said was not sufficient to make ends meet. “Even when I have a good day and make some money, I can’t decide if I will use it to buy food or pay the bills. It’s impossible to survive on these meagre amounts.”111

Katarina lives with her family of five, including three small children. She also lost her assistance of 24,000 Serbian dinars a month (200 euros) in May 2023 because her husband made about 15,000 Serbian dinars (120 euros) from selling scrap metal. Katarina’s two teenage sons had to work four full days collecting empty bottles, crushing and selling the glass just to make about 1,000 Serbian dinars (about 8.50 euros): 

We would prefer to work instead of relying on the assistance. My son finished secondary school, but he hasn’t been able to get any work. He applied many times, including at the new factory that was recently built in Kraljevo, but he never gets called. It’s impossible to find a job and we are left to rummage through containers and landfills to survive.

Katarina from Kraljevo

Katarina’s family was among many in Kraljevo who either lost social assistance or had it reduced due to their modest earnings being flagged by the Social Card.112 

“This new procedure caused chaos in the village. Most people lost assistance,” said Miloš, a middle-aged man from Kraljevo. Like many others, he was disheartened by now having to choose between receiving a meagre but regular social assistance and doing occasional informal work that made him feel more independent: 

Two years ago, the Centre for Social Work and the mayor met with Roma representatives from the area and told us that we would be better off earning a proper income than relying on government handouts. And now they have removed almost the entire village from the assistance that was already too low because we tried to make some money by selling scrap metal to feed our families. Yet we can’t get regular work.

 Miloš, 24 April 2023, Kraljevo, Serbia.

Others continue to take the risk. Danica, the young single mother with a daughter aged six who has learning difficulties, said she didn’t have a choice. “I live in squalid conditions. I have no running water in my house, no bathroom. My electricity is also unreliable because I am connected to my neighbour’s house.” Danica receives 13,000 Serbian dinars (110 euros) in social assistance, but because this is far from enough to live on, she continues to go to the neighbouring landfill to collect bottles. “I have to take my daughter along because I can’t leave her with anyone, but it’s not safe to take her there. That’s not a place for a child. I have to keep an eye on her all the time. But I don’t have an alternative. I can’t survive on social assistance alone.”113

The minimum income conditions for social assistance are determined by the Law on Social Protection and related government decrees that have been in place since 2011. The prohibition of combining small earnings with social assistance is already penalizing families living in poverty for trying to bridge the gap between inadequate social assistance and the actual cost of living. However, introducing the Social Card registry has made the already restrictive criteria absolute and changed the role of social workers by removing the small discretion they could previously exercise to evaluate whether people’s supplemental income from informal work had a meaningful impact on their socio-economic standing. This has pushed many families deeper into poverty and forced others to make unacceptable trade-offs by having to choose between social assistance and a small income, neither of which on their own is sufficient to cover their basic needs.

Footnotes

[109] Amnesty International interview with staff of Centre for Social Work, 27 April 2023, Kraljevo. 

[110] Amnesty International interview with Edna, 24 April 2023, Kraljevo, Serbia.

[111] Amnesty International interview with Milan, 24 April 2023, Kraljevo, Serbia.

[112] Amnesty International interview with Katarina 24 April 2023, Kraljevo, Serbia.

[113] Amnesty International interview with Danica, 24 April 2023, Kraljevo, Serbia.


Consequences of flaws in the social card registry: The destabilizing effect of the loss of social assistance

For people living in poverty, even modest but regular social assistance and other related support, such as access to a soup kitchen, can mean the difference between feeding their families and going hungry. Sudden loss of or disruptions to social assistance not only severely impair people’s ability to provide for their families, but also to budget effectively. “It’s really difficult to get food and to take care of the children. It’s hard. We live from day to day. Some days we have food, others we don’t,” Katerina told Amnesty International.114

Being removed from social assistance after the Social Card registry was introduced has hit families with children exceptionally hard and also severely impacted their ability to exercise their other social and economic rights. Bogdan from Belgrade has four young children, two of whom are babies: 

They need baby formula and diapers. We always lived modestly and without any luxuries, but now we can’t even buy food. This is particularly hard for my children who are in school. We used the money from social assistance to get food, hygiene supplies and clothes so they can look like other children and not get bullied because they are Roma. Now I’m not sure if we can keep sending them to school. We literally can’t give them a snack to take to school. Education is very important to us. I want our children to go to school and have a better life, but without food and proper hygiene, it’s very difficult.

Bogdan 25 April 2023, Belgrade, Serbia

Losing their assistance, including soup kitchen access, has deprived many children from poor families of their only daily cooked meal and immediately plunged families into serious hardship. Edna told Amnesty International: 

Life was much easier when we were receiving regular monthly social assistance. When we lost it, we were left with nothing. What my husband makes [by selling scrap metal] is not enough to feed our family, get medication or send the kids to school, especially now that the prices have gone up.  

Edna, 24 April 2023, Kraljevo, Serbia

For Ratka, a woman from Belgrade who lives with her partner and four children, losing social assistance made their already difficult situation unbearable. “It was difficult to survive with the assistance – which I really appreciated – but it is extremely low. I used to get 27,000 dinars (230 euros) [per month] for my family, but one weekly shop for essentials in Lidl costs 7,000 dinars (60 euros). What about the rest of the days?!”115

Jelena is a single mother with multiple health problems. She and her son, aged five, were left without social assistance because she was erroneously flagged by the system as having an income placing her above the eligibility threshold. This happened despite Jelena showing social workers proof of her actual income. However, the disruption in her social assistance payments left her struggling to meet her own and her son’s needs and severely impacted her mental health. “All this brings me down, both physically and emotionally,” Jelena told Amnesty International.116

Vanja is an older woman who also has significant health issues and a mobility problem. She had been without social assistance for six months when Amnesty International interviewed her. She described how difficult this was for her whole family. “Women in my neighbourhood give me a plate of food to eat occasionally. Otherwise, I have nothing. I look at my children, who are hungry. It’s terrible to have to watch your children starving.” Vanja said many families in her neighbourhood had similar experiences. “I know families who have to send their children to beg because they have nothing to eat, especially now that everything is more expensive.”117 After receiving legal support from a Serbian NGO, Vanja was successful in her appeal. 

The loss of and sudden disruptions in benefits had a highly destabilizing effect on the people Amnesty International interviewed, who already faced economic precariousness. In addition to unexpectedly losing their benefits due to the Social Card registry flagging new information, some already faced a three-month gap in receiving social assistance, as noted above. For people already struggling to cover their living costs even with social assistance, the gap in payments often forces them to borrow money to pay bills or buy food. Many end up in debt that they cannot pay off, leaving them in a permanently vulnerable state. In its Concluding Observations on Serbia in 2014, the CESCR stated that these interruptions in social assistance were unjustified”.118

The problems highlighted in this report show that introducing the Social Card registry with its use of automation has compounded the existing deficiencies in Serbia’s social security system. It has also jeopardized people’s right to social security and their exercise of other related economic and social rights. Relying on raw, often inaccurate or unreliable source data from databases that are not regularly updated or data that does not accurately reflect people’s current economic hardship to determine their eligibility for social assistance clearly undermines the requirement that eligibility conditions should be “reasonable, proportionate and transparent”.119

The speed with which people lose social assistance and the lack of clarity about the reasons behind the loss is also at odds with the CESCR recommendation that “the withdrawal, reduction or suspension of benefits should be limited and based on grounds that are reasonable, subject to due process, and provided for in the national law.”120

Footnotes

[114] Amnesty International interview with Katarina 24 April 2023, Kraljevo, Serbia.

[115] Amnesty International interview with Ratka, 25 April 2023, Belgrade, Serbia

[116] Amnesty International interview with Jelena, 25 April 2023, Belgrade, Serbia

[117]  Amnesty International interview with Vanja, 25 April 2023, Belgrade, Serbia.

[118]  Amnesty International interview with Vanja, 25 April 2023, Belgrade, Serbia.

[119] UN CESCR, Concluding Observations on the second periodic report of Serbia, 10 July 2014, UN Doc. E/C.12/SRB/CO/2, paras 24 (a) and 24 (b).

[120] UN CESCR, General Comment 19: The Right to Social Security, 4 February 2008, UN Doc. E/C.12/GC/19, para. 24.


Are social workers still autonomous decision-makers?

Introducing the Social Card registry has not only increased the importance of accurate data, but it has also simultaneously reduced applicants’ room to properly engage with social workers who are better able to recognize the complexity of their individual situations than the registry is. It has also reduced the possibility of preventing such errors, or meaningfully reviewing or correcting information about individual situations in cases where there is a discrepancy between data held by the registry and reality. 

The Ministry of Labour told Amnesty International that the Social Card registry does not make automated decisions based on entry-level data. Instead, the system merely flags discrepancies and changes in recipients’ status and instructs social workers to “engage in an administrative process”; that is, to investigate to ascertain if the discrepancy could be easily resolved.121

The Ministry also insisted that decision-making is the “autonomous responsibility” of social workers.122 However, our research found that social workers often did not know how to explain any discrepancies in data held by the registry to recipients; were reluctant to update or correct erroneous information or did not know how to; and lacked clarity about how to address notifications flagging a discrepancy in a case.123 Indeed, in correspondence with Amnesty International, the Ministry of Labour maintained that social workers did not need to know how the system works in order to do their job.124 Most of the people Amnesty International interviewed said social workers attributed decisions to the “new system.” They also insisted that they could not contradict or override notifications pointing to clear errors. “There is nothing I can do, it’s the new system from Belgrade that decided,” was a frequently documented response from social workers noted in interviews.125 In conversation with Amnesty International, some social workers maintained that they were not able to override the notifications because it was not technically possible to do so. Others said they were reluctant to contradict the notifications for fear of being sanctioned for exercising discretion as the Ministry of Labour now had the technical ability to oversee all activities in the Centres for Social Work. Staff in one Centre for Social Work visited by Amnesty International expressed concern about being surveilled and micro-managed through the system, noting that this discouraged social workers from making autonomous decisions on applications, especially if the Social Card registry flagged discrepancies that were in their judgement, errors.126

The Ministry of Labour also insisted that the streamlined collection of documents introduced by the Social Card registry freed up social workers’ time and created more opportunities for direct interaction with recipients.127 However, interviews with social workers and recipients of social assistance contradict this. The findings suggest that whilst social workers maintained that field work and interaction with applicants and recipients was the core of the assessment process, the Social Card registry, in many ways, deprioritized field visits. Social workers told Amnesty International that they investigated cases flagged by the new system but frequently deferred to the data in the registry even when they ascertained that they were incorrect, in part due to automation bias and in part because they were not able to change the data in the system.128

Moreover, the lack of training provided to social workers on the risks of automation bias means that the introduction of the Social Card registry has potentially compromised the autonomy of social workers’ decision making. 

Given the possibility of unreliable or inaccurate data being included in and used by the Social Card registry, the limited role of social workers, as well as the serious implications of these processes on the lives of individuals, the Social Card registry is not fit for purpose in its current design. 

Footnotes

[121] Official correspondence by Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[122] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[123] Based on interviews and conversations with social workers in Serbia in November 2022 and April 2023.  

[124] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International.

[125] Based on interviews with people who lost social assistance in Kraljevo and Belgrade, Serbia, April 2023.

[126] Interview with social worker in a Centre for Social Work, April 2023, Serbia.

[127] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International.

[128] Interview with social worker in a Centre for Social Work, April 2023, Serbia.

an illustration of a woman drowning in paperwork

Chapter 6 – The discriminatory effect of the social card registry  

Introducing the Social Card registry without addressing existing barriers to accessing social assistance was bound to create new challenges and exacerbate existing problems for individuals already facing discrimination. At face value, the Social Card Law and the Social Card registry may be presented as neutral technical solutions. However, in practice, they cannot be isolated from the social and historical contexts into which they are introduced. Amnesty International’s research found this to be a case in which flawed technology was introduced in a context where marginalized groups were already experiencing discrimination. 

Laws in Serbia, including the Social Card Law, do guarantee formal equality for all individuals. Nevertheless, the practical implementation of the Social Card Law and the Social Card registry does not provide substantive or de facto equality.129 As outlined above, the Social Card registry apparently ignores the specific needs of already marginalized individuals and communities, as well as the barriers they face. This puts Serbia at risk of not respecting its obligations on the right to equality and non-discrimination. In turn, this undermines states’ obligation to “enact special measures to achieve and protect racial equality throughout the public and private spheres.”123

The Social Card registry relies on an integrated data-driven model to function and so the design choices around what data is collected and from what sources. These are not neutral decisions and, without the necessary nuances and safeguards, can, therefore, result in discrimination. For example, choosing to collect and process extensive data on social protection applicants and recipients, as well as on their “related persons”, opens up the possibility for surveillance of individuals on lower incomes and those who experience several intersecting forms of marginalization.  

Gaps and imbalances in data processed by automated or semi-automated systems can similarly lead to discrimination. A social worker told Amnesty International that before the Social Card registry was introduced, and especially when working with marginalized communities like Roma, social workers knew that some data was inaccurate or out of date. For example, multiple cars registered to someone living in extreme poverty would not be considered important assets for social assistance eligibility but rather vehicles likely sold for scrap metal or that otherwise no longer existed.131

Experience from other countries which have introduced social registries showed that comprehensive and ongoing outreach targeting specific communities was crucial to ensure their access to benefits. The authorities in Serbia did not conduct targeted information campaigns, nor did they prepare recipients, especially those from marginalized communities, for the changes that could affect their eligibility for social assistance or explain how the new programme operates. This has left members of the Roma community particularly vulnerable to the loss of financial assistance for failing to update their records on car ownership or engaging in informal work. 

Before Serbia introduced the Social Card registry, social workers played a more significant role in verifying applicants’ data and documents by conducting field visits and interviews. This allowed for a more meaningful assessment of people’s living conditions. While human data verification is important in any system, it is vital when dealing with already marginalized communities facing multiple – possibly invisible – factors of deprivation. The new system and its automated components have, in effect, reduced social workers’ role as many now defer to the data contained in the registry because they are reluctant to or do not know how to investigate and validate the information flagged. 

Serbia’s Ministry of Labour insisted that laws governing social security, including the Social Card Law, did not treat Roma or any other marginalized groups differently. The Ministry also claims that it has the legitimate right to use “true and accurate data which are necessary for the enjoyment of social security rights”.132 The Ministry did not recognize the fact that the seemingly innocuous and objective datasets being used as indicators of socio-economic status often ignored the specific context of a community’s marginalization, such as their living conditions, barriers to employment, and their particular needs, as the UN CESCR has repeatedly highlighted in its previous reviews of Serbia.133

Due to Serbia’s historical and structural context, many individuals from marginalized backgrounds have persistently low literacy and digital literacy levels. They, therefore, face challenges when interacting with administrative departments to keep their paperwork up to date or to appeal their removal from the social assistance system. In this way, the Social Card registry represents yet another barrier to accessing social assistance, which can amount to indirect discrimination. 

Amnesty International’s research found that the Social Card registry is not designed to factor in the challenges and barriers faced by those communities most critically dependent on social assistance, including Roma, people with disabilities and women. Women, who are represented across all groups, are more likely to receive social protection and may also face additional intersectional barriers to accessing their rights. 

Some of these challenges partly stem from the fact that the Social Card registry was not developed based on full and active consultation with affected communities. Additionally, no special measures, including targeted information campaigns and comprehensive outreach to these communities to explain how the new system would affect them, were taken to ensure that individuals and communities facing multiple and intersectional forms of discrimination could access their right to social assistance.134 Serbia is, therefore, not only failing to remove obstacles to people’s realization of their rights but also creating an additional barrier by introducing the Social Card registry. This may, in effect, amount to indirect discrimination. 

Footnotes

[129] UN CERD, General recommendation 32, The meaning and scope of special measures in the International Convention on the Elimination of All Forms [of] Racial Discrimination, 24 September 2009, UN Doc. CERD/C/GC/32.

 [130] UN CERD, General Recommendation 32, The meaning and scope of special measures in the International Convention on the Elimination of All Forms [of] Racial Discrimination, 24 September 2009, UN Doc. CERD/C/GC/32, para. 23.

[131] Amnesty International interview with a social worker, 27 April 2023, Serbia.

[132] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.  

[133] UN CESCR, Concluding observations on the third periodic report of Serbia, UN Doc. E/C.12/SRB/CO/3

[134] UN CERD, General Recommendation 32: The meaning and scope of special measures in the International Convention on the Elimination of Racial Discrimination, 24 September 2009, UN Doc. CERD/C/GC/32.


Serbia’s Roma community

“Roma seem left alone and forced to steal and lie to survive, just like the stereotype here has it! But I might not have a choice if I want to feed my family,”135 Bogdan told Amnesty International in Belgrade. 

The most recent census data shows about 147,000 ethnic Roma living in Serbia, making up a little over 2% of the country’s overall population. However, unofficial sources estimate the real population to be significantly larger, ranging between 250,000 and 500,00 people. Roma are among the most marginalized communities in Serbia. They face deep systemic discrimination in all aspects of life, from accessing health and housing to education and employment opportunities.136 

The 2019 Multiple Indicator Cluster Survey (MICS) for Serbia found that five out of six Roma households, or 83%, live in conditions of pronounced material deprivation compared to about 40% of their non-Roma neighbour. Roma families are also disproportionally represented in Serbia’s social security system, with MICS estimating that over 60% of Roma receive financial social assistance. This is almost four times higher than for Serbia’s total population. Data shows that social assistance is the only income for over 25% of Roma households

An intersectional lens is crucial to understanding the discriminatory impacts of the Social Card Law, because many individuals experience “situations of double or multiple discrimination”.137 Roma children and women are particularly exposed to multiple factors of deprivation. A regional study in the Western Balkans showed that over one-third of young Roma people lacked basic education. They were also twice as likely to lack education, training or employment than neighbouring non-Roma youth. Such poor early outcomes have life-long implications, impeding one’s chances of finding work later in life. For Roma women, nine in ten are not in employment, two in three lack any funds of their own, and one in two do not have pension coverage. The UN CERD recommended that states specifically take Roma women’s needs into account when developing social assistance programmes because they often face both racial and gendered discrimination.138 

The disproportionately poor societal outcomes for Roma compared to non-Roma people and the percentage of Roma families living in poverty point to systemic discrimination in Serbia. Indeed, in April 2022, the UN CESCR noted its concern about the “substantive discrimination faced by disadvantaged and marginalized individuals and groups in accessing work, housing, and education”.139

Roma or internally displaced people (many of whom are also Roma displaced from Kosovo) often live in informal settlements, lack identity documents and may not have their administrative affairs in order. This creates considerable obstacles to accessing social protection. Even before the Social Card registry was introduced, Roma’s access to the social security system was impeded in many ways, including a lack of accessible information about available services, inadequate Centre for Social Work capacity and structural discrimination.140 Some eligibility requirements, such as capping child allowance at a maximum of four children per family and the condition that children in families receiving social assistance must regularly attend school,141 already had a discriminatory effect on Roma and excluded many families living in poverty

Because many internally displaced Roma still lack identity documents, live in informal settlements in precarious conditions, have high rates of functional illiteracy, face current and historical discrimination and are treated with suspicion by the authorities, they often do not or cannot readily engage in administrative processes. Introducing an automated data-driven system, therefore, favours people who are able to maintain good record-keeping and puts Roma at a particular disadvantage. 

Footnotes

[135] Amnesty International interview with Bogdan, 25 April 2023, Belgrade, Serbia.

[136] Perisic, Natalija and Vidojević, Jelena, “Divided by poverty and social exclusion – Roma and persons with disabilities in Serbia”, 2015, pp. 142-159.

[137] UN CERD, General Recommendation 32: The meaning and scope of special measures in the International Convention on the Elimination of Racial Discrimination, 24 September 2009, UN Doc. CERD/C/GC/32, para. 8. 

[138] UN CERD, General Recommendation XXVII on Discrimination Against Roma, 16 August 2000, A/55/18, annex V.

[139] UN CESCR, Concluding observations on the third periodic report of Serbia, 6 April 2022, UN Doc. CCPR/C/SRB/CO/3, para. 28.

[140] Friedrich-Ebert-Stiftung, “Social dimension of Serbia’s accession to the EU”, p. 15.

[141] Roma Children face persistent barriers in accessing education. “There are three main problems in Roma education, identified by all of the main strategic documents: Roma are not fully included in the educational system, they are not receiving quality education and they are often exposed to discrimination and segregation.” Furthermore, “[t]here is still a strongly formulated belief that exclusion from the educational system is a matter of choice, putting thus the greater responsibility upon the Roma community. This belief is formed by the stereotypes and prejudice that result from ignorance.” Perisic, Natalija & Vidojević, Jelena, “Divided by poverty and social exclusion – Roma and persons with disabilities in Serbia”, 2018, 142-159, pp. 150.


People with disabilities and health conditions

The most recent census data identifies 8% of the Serbian population as people with disabilities. The Regional Cooperation Council, however, puts this figure at an estimated 10-15%.142 People with disabilities in Serbia face discrimination in accessing their rights to work and education and to an adequate standard of living.143 Only 9% of people with disabilities are in employment, while this figure drops to just 4% for women with disabilities. Of people with disabilities in employment, only a third said that their workplace had adapted to their needs.144 12.2% of people with disabilities did not attend primary school, a rate significantly higher than the 2.7% for the general population.145 In light of such figures, it is no surprise that people with disabilities in Serbia are among the groups most at risk of experiencing poverty.146 Serbia’s National Strategy for Sustainable Urban Development includes measures to reduce poverty for people with disabilities by 2030. However, it also recognizes that a lack of resources presents a significant threat to the social inclusion of and poverty reduction for people with disabilities.147 Even when laws and policies include references to inclusion or a human rights-based approach to disability, “their practical application is missing”, with multiple structural barriers keeping people in a “constant circle of poverty and social exclusion”.

There are significant barriers to establishing reliable, inclusive data on people with disabilities in Serbia. This can be partly linked to the lack of a consistent and universal definition of disability across various sectors, with people with psychological and mental health issues, as well as other health conditions, often being overlooked. A recent country report on Serbia by the UN Partnership on the Rights of People with Disabilities (UN PRPD) recommended that the Serbian government improve inclusive data collection on people with disabilities.267  The UN CRPD has also raised concerns about the Serbian authorities’ failure to collect and analyse data on the situation of persons with disabilities that provide sufficient information to implement the Convention.148

Both the UN PRPD and the UN CRPD recommend amending national anti-discrimination legislation to recognize both intersectional discrimination and the denial of reasonable accommodation as a specific form of disability-based discrimination. 

People interviewed by Amnesty International highlighted the additional barriers they face due to their own or a family member’s disability or health condition. For many people, their disability or a health condition that may not be recognized as a disability under Serbia’s laws was an additional barrier to accessing employment. Buying medication and other healthcare treatments was also an additional significant strain on their already tight budgets. Katarina from Kraljevo told Amnesty International that she and several members of her family had multiple chronic health conditions requiring medication and treatment   – the cost of which they had to cover with their social assistance payments. She told Amnesty International that just one inhaler – a medical necessity– cost 3,000 Serbian dinars (25 euros). This placed a significant burden on their household budget as they only received 24,000 Serbian dinars (204 euros) in monthly social assistance.149

Milan, also from Kraljevo, told Amnesty International that his medical condition limited the work he was able to do and was therefore reliant on social assistance to survive. He had so far not been able to find secure employment despite having applied for multiple jobs.150 When the Social Card registry flagged Marijan for earning a small and irregular income from selling scrap, he was removed from the system, plunging him and his family into poverty. 

Danilo from Novi Sad receives both social assistance and disability benefits due to a debilitating back injury. Compared to people who only receive financial assistance, Danilo is in a slightly better financial position, and yet he can barely make ends meet. He lives in a small studio apartment without running water or electricity and shares a bathroom with residents on the same floor. A neighbour allows him to connect a small lamp to his electric wire at night and charge his mobile phone. “Once I pay the utility bills (communal council tax) buy food and all my medication, I am left with absolutely nothing before the month is out. I end up borrowing money from friends, but then I have to pay them back next month, and I am constantly behind,” Danilo said.151

Contrary to the CRPD, most disability assessments in Serbia are based on the medical model of disability, with a focus on an individual’s “degree of disability” rather than their need for support.152 Serbia’s Law on Social Protection sets an unusually high threshold for those who are unable to work, categorizing many people with mental health problems and some degree of physical disability as “able-bodied”.153 In these cases, people are eligible for nine months of social assistance, which, in theory, leaves them with the possibility of getting temporary or seasonal employment in the remaining three months. This leaves people who are already in a vulnerable position and unable to work without any assistance for at least three months each year. 

While new technological efforts such as the automated Social Card registry may aim to streamline processes, the lack of proper safeguards renders many people susceptible to scrutiny. As already outlined, gaps and imbalances in data processed by automated systems to reach a conclusion about eligibility can also lead to discrimination. Despite asking the Serbian authorities for more detailed information on how they ensured that the Social Card registry respected the right to equality and non-discrimination, Amnesty International has not found adequate evidence that the Serbian authorities have taken appropriate measures to ensure that the new Social Card Law did not lead to discriminatory outcomes.154 These measures would include considering the impact of the Social Card registry on disadvantaged individuals’ and groups’ enjoyment of economic, social and cultural rights and preventing harms from occurring, and ensuring that there is no potential for bias or discrimination in the way data is collected, processed or otherwise used.   

Footnotes

[142] UN PRPD and UN Serbia, Situational Analysis of the Rights of Persons with Disabilities in Serbia, September 2023, https://unprpd.org/sites/default/files//library/2023-09/CR%20Serbia%202023.pdf; Regional Cooperation Council, Professional rehabilitation and employment of persons with disabilities in Serbia, Regional Cooperation Council, Host Country Case study, 2018.

[143] In addition to general human rights standards on the right to work, the right to education, and the right to an adequate standard of living, additional standards protect the rights of persons with disabilities. This includes the Convention on the Rights of Persons with Disabilities (CRPD), in particular Articles 27 (Work and employment); 24 (Education); and 28 (Adequate standard of living and social protection). 

[144] Regional Cooperation Council, Professional rehabilitation and employment of persons with disabilities in Serbia, Host Country Case study, 2018.

[145] Beker, K., Janjic, B., “Covid-19 measures and social and economic rights of women with disabilities in Serbia”, Out of Circle Belgrade & FemPlatz, 2021.

[146]  Beker, K., Janjic, B., “Covid-19 measures and social and economic rights of women with disabilities in Serbia”, Out of Circle Belgrade & FemPlatz, 2021

[147] Official Gazette of the Republic of Serbia, no. 47/2019

[148] UN CRPD, Concluding observations on the initial report of Serbia, 23 May 2016, UN Doc. CRPD/C/SRB/CO/1, para. 63. 

[149] Amnesty International interview with Katarina 24 April 2023, Kraljevo, Serbia.

[150]  Amnesty International interview with Milan, 24 April 2023, Kraljevo, Serbia. 

[151] Amnesty International interview with Danilo, 29 September 2023, Novi Sad, Serbia.

[152] European Commission, Task 2017-18 Disability assessment – country report Country: Serbia.

[153] Serbia, Law on Social Protection, Official Gazette of RS number 24/11.

[154] The test which compares a treatment of people who have protected characteristics, such as race, religion, ethnicity, etc. with the treatment of those who do not have the same protected characteristics in order to determine potential discrimination

an illustration of a woman sitting in a chair looking at her young child playing with toys. There is a pile of bills on her lap

Chapter 7 – The right to remedy  

Lack of clarity and a complex appeals process

The system is absurd. It seems put together to kick people off assistance, rather than help them. They make the paperwork so difficult that I am about to give up fighting  
for it.

Bogdan, a man aged 32, interviewed by Amnesty International

The Serbian authorities are responsible for communicating with individuals about their right to appeal any decisions made by a public body. Serbia also needs to ensure that the mechanism for appealing decisions about social assistance is clear, transparent and accessible to all, including people who are unable to access resources online or who have low levels of literacy.155 The appeals process should be timely, avoid leaving individuals and their families without access to support, and respect their right to an adequate standard of living while their appeal is being processed.156

The Serbian Ministry for Labour told Amnesty International that recipients, including those who have lost social assistance, receive information about their rights and legal procedures pertaining to appeals “in writing, verbally, in centres, in the ministry, by phone, email or voice messages”.157 

In practice, however, the mechanism for appeals and reviewing cases under Serbia’s relevant legislation does not provide for a meaningful and effective remedy.158 People who lose social assistance have 15 days to file an appeal, a time frame often too short to allow them to gather all the supporting documentation required to successfully file an appeal. Due to the opacity of the system and without full information about how decisions were made and based on which data, recipients are often reluctant to engage in a lengthy and uncertain administrative process to reinstate their assistance. If people miss the 15-day window, they are forced to wait three months before they can reapply for support, regardless of their eligibility or need. 

Most people Amnesty International interviewed shared their frustration with the appeals process. When the Social Card system was introduced in Belgrade, Ratka was removed from social assistance due to discrepancies in her Social Card registry data despite having presented evidence to support her claim. A social worker told her that they could not adjust data received by the registry and instructed her to amend the error in the originating database. This required Ratka to travel in person to the different department offices holding the various databases and submit proof that their data was out of date or otherwise erroneous. This was a time-consuming process at a time when Ratka aced intense pressure caused by losing her social assistance. She told Amnesty International that “this new system seems rigged against people. They only tell you what’s on the surface and what they see on the screen, but you have no idea what information goes into it or how you can fix mistakes”.159

Edna, a woman aged 40, lives in Kraljevo with her husband and six small children in a 20-person extended household. She told Amnesty International that she was informed over the phone that she was no longer eligible for social assistance but that she did not receive information on how to appeal. As a result, she missed the 15-day deadline for appeals and had to wait three months to restart the process.160

Dinko lives with his parents, his wife and their five children. In March 2023, Dinko received a call from the Centre for Social Work saying that his data Social Card registry data showed that he and his family were no longer eligible for social assistance. With support from a local Serbian non-governmental organization (NGO), he collected information from the municipality and the police station to prove his eligibility. The Centre for Social Work accepted this documentation and reinstated his household’s social assistance.161 Dinko said he felt that his “life depends on the decision of the social centre”. 

Dinko was one of a few fortunate people who managed to get their social assistance back because of free legal support from an NGO. In the absence of an accessible appeals process or government support, many people rely on assistance from civil society organizations to navigate the complex administrative process and collect evidence for a successful appeal. Despite the positive work that these organizations do to help people realize their rights, a lack of resources and capacity prevents them from providing legal aid to all those who need it. However, it is the state that is responsible for ensuring that people are supported throughout the appeals process instead of relying on local NGOs to fill the gap. 

The difficulties Dinko faced in gathering evidence to correct out of date information and errors in the Social Card registry are all too common. Others Amnesty International spoke with recounted numerous challenges in trying to collect documentation and appeal decisions on social assistance. 

Bogdan lives with his wife and four very young children. He was removed from social protection after being told that the Social Card registry flagged him for having 95,000 Serbian dinars (810 euros) in a bank account and a car registered in his name. However, in reality, Bogdan did not have a bank account, and the car in question was sold for scrap metal in 2020. Reflecting on the appeals process, Bogdan told Amnesty International: “The system is absurd. It seems put together to kick people off assistance rather than help them. They make the paperwork so difficult that I am about to give up fighting.” 

Brane from Belgrade has a wife and four small children. They lost social assistance in March 2023 and spent weeks trying to understand why they were cut off from the system. A social worker in the Centre for Social Work told Brane that the Social Card registry found income in his bank account, which put him over the threshold for assistance. However, she was not able to give him any more information or instruct him on how to rectify this. After approaching the A11- Initiative for Economic and Social Rights, an NGO, for assistance, Brane went to the Pension Insurance Fund to get confirmation that he was not gainfully employed and to the Tax Administration to get a note about his “income”. After spending several weeks going from office to office, he finally received confirmation that he did not have funds in any bank account and that this was indeed a Tax Administration mistake. Nevertheless, Brane missed his appeals window and was forced to wait three months to reapply for social protection, during which time he and his family had no support.162

In some cases, people Amnesty International spoke to noted that the Centre for Social Work did not provide them with a clear reason for removing their social assistance and that this was in itself a barrier to them submitting their appeal. In other cases, people said that they did not receive a decision in writing and were not told that they could appeal against it. Katarina from Kraljevo, who was removed from social assistance along with her husband and three children, said: “I was told that the assistance would be stopped because we had money in our bank account. I didn’t understand what they were talking about, and they didn’t explain anything”. She said she did not receive any written documentation or information about her right to appeal the decision. 163

Several people told Amnesty International that they were informed by the Centre for Social Work that the appeals process could take a long time. Rather than trying to appeal the decision, they were advised to wait for three months and reapply. In Kraljevo, many people who lost social assistance during the first phase of the implementation of the Social Card Law opted to wait and apply again. This left them and their families without any state financial support in the meantime. Other applicants said social workers advised them to restart their application rather than challenge a negative decision. This left people who were already marginalized and living in poverty with little to no savings, with no choice but to go without critical support for several months to secure another chance to get social assistance. 

The fact that it is easier to reapply for social assistance than it is to appeal a decision, points to a serious flaw in Serbia’s social protection system and has serious ramifications for the rights to remedy and redress. By not giving people a meaningful opportunity to effectively challenge decisions on social assistance, Serbia is not upholding its obligations under international and domestic law.  

As outlined above, opaque Social Card registry decision-making processes, the lack of clarity about the appeals process, the burden of gathering supporting documentation placed on individuals, and the unreasonably short timeline for appeals all unduly undermine people’s right to remedy. 

The right to remedy is especially important when it comes to using digital technologies in the public sector. There are core risks related to the fact that these systems often operate in opaque ways. This makes it hard to understand how they work, let alone challenge the resulting decision-making. States are, therefore, responsible for ensuring that any use of technology is clearly communicated to rights holders and for ensuring transparency so that the outputs of the technology and the decision-making process are visible and interpretable. Interpretability refers to whether humans can easily understand the functions and outputs of the technology. Finally, states must “create accessible and practical routes for remedy and redress when human rights are negatively impacted”. 

The Serbian government has signed and ratified CoE Convention 108+ for the protection of individuals with regard to the processing of personal data. Furthermore, Serbia’s Data Protection Law, which came into force in 2018 and governs data protection, is largely harmonized with the EU’s General Data Protection Regulation. The Serbian Law on Personal Data Protection governs the complaints procedure when a data subject believes their data rights have been breached.164 Individuals with concerns related to the quality, accuracy, or use of their data by the Social Card registry could file a complaint with the Data Protection Commissioner. Any person experiencing material or non-material damage also “has the right to receive monetary compensation for damage”.165 However, to access their data, individuals have two options: they can either use an e-government portal or submit a request via email or post to access their data in person.166 The first option relies on a high level of digital literacy. The second option requires travelling to Belgrade in person, which would be prohibitive for many people both due to cost and the time it would take. Two Serbian NGOs familiar with communities receiving social assistance have identified the current appeals process as a significant barrier for people lodging a complaint.167

Serbia’s Social Card Law raises concerns about asymmetries in information, which is when the state has more information on how a system works than individuals and their advocates do. This presents a barrier to people understanding and potentially challenging such systems when their rights are impacted. Amnesty International’s 2021 report, Xenophobic machines: Discrimination through unregulated use of algorithms in the Dutch childcare benefits scandal, highlighted a lack of transparency about families being denied childcare benefits after being investigated for fraud as a key and ongoing concern. Many families still have no answers as to why they were investigated for fraud. The opaque decision-making process within the algorithmic system in this case, including how human operators use the system and the lack of information provided to people affected, makes efforts to investigate and hold the system to account for human rights violations more difficult. It also places the burden of proof on impacted communities and NGOs representing their rights. The UN Special Rapporteur on extreme poverty and human rights has raised concerns about how, in the context of a digital welfare state, “determinations are framed and communicated” that may “be dehumanized and allow no room for meaningful questioning or clarification”.168

Footnotes

[155] UN CESCR, General Comment 19: The right to social security (Art. 9 of the Covenant), 4 February 2008, UN Doc. E/C.12/GC/19. 

[156] In order to comply with international human rights law, “[s]ocial protection programs must incorporate accessible and effective complaints mechanisms which guarantee anonymity, allow for individual and collective complaints, and are sufficiently resourced and culturally appropriate. Complaints procedures should include an appeal process that is independent, accessible, simple, fair and effective”. Magdalena Sepúlveda and Carly Nyst, The Human Rights Approach to Social Protection, 2012. 

[157] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[158] The appeals process is governed by the Law on Administrative Procedure.

[159] Amnesty International interview with Ratka, 25 April 2023, Belgrade, Serbia.

[160] Amnesty International interview with Edna, 24 April 2023, Kraljevo, Serbia.

[161] Amnesty International interview with Dinko, 25 April 2023, Belgrade, Serbia.

[162] Amnesty International interview with Brane, 25 April 2023, Belgrade, Serbia.

[163] Amnesty International interview with Katarina 24 April 2023, Kraljevo, Serbia.

[164] Serbia: 2018 Law on Personal Data Protection 13 November 2018. 

[165] Serbia: 2018 Law on Personal Data Protection 13 November 2018, Article 86.

[166]  Interview with the Office of the Data Protection Commissioner, 28 April 2023, Belgrade, Serbia

[167]  Interview with European Roma Rights Centre, 29 April 2023, Belgrade; Interview with A11 Initiative, 27 April, 2023. 

[168] UN Special Rapporteur on extreme poverty and human rights, Report: Digital technology, social protection and human rights, UN Doc. A/74/493, para. 55

An illustration of a group of people standing in front of a giant lady of justice, who has fallen on her side.

Chapter 8 – Lack of accountability  

The authorities’ failure to conduct adequate risk assessments and monitor implementation

Digital technologies, while often presented as neutral, “can reflect values and assumptions that are far removed from, and may be antithetical to, the principles of human rights”.169 Examples of this include using data to assess eligibility for social assistance in Serbia’s Social Card registry or discriminatory algorithmic decision-making to detect alleged childcare benefit fraud in the Netherlands. Adequate and robust human rights safeguards are therefore essential for preventing human rights harms from occurring after introducing these technologies into the public sector. This includes but is not limited to, the need for authorities and other policymakers to undertake a full human rights risk assessment before deploying a new system. They must also conduct ongoing monitoring to ensure that the system causes no harm throughout its implementation and lifecycle and adopt special measures to address discrimination. Crucially, any system found to potentially cause human rights harms or discriminatory outcomes in its early design and conceptualization stages should not be deployed. 

In official correspondence with Amnesty International, Serbia’s Ministry of Labour said that an analysis of the Social Card Law adoption was carried out in “the legally prescribed procedure and that no risk to the rights of users was identified”.170 According to the Ministry, the impact assessment on processing personal data within the Social Card registry identified a “low and very low level of risk to personal data” and that “adequate protection measures were taken to minimize” this risk.171 However, the Ministry of Labour has to date not made public any comprehensive human rights risk assessment of the Social Card registry undertaken at any stage. Nor has the Ministry provided public access to any ongoing assessments to monitor the registry’s impact on human rights in its implementation phase. Access to disaggregated data on the types of cases being flagged in the Social Card registry as requiring investigation by a social worker would help to better understand the characteristics of individuals flagged and thereby establish any discriminatory trends. 

At present, it is not clear what was included in the Ministry’s risk assessment and whether the authorities exercised full due diligence to properly identify and address potential human rights harms. What is clear is that government agencies and civil society organizations alike have raised concerns about the use of automation in the Social Card registry, the reliability of the data, the scope of data processing, the ability of social workers to override the system and the potential for the system to exclude or otherwise harm marginalized groups. 172

In a meeting with Amnesty International, representatives from Serbia’s Data Protection Commissioner’s office said that they participated in consultations with the Ministry of Labour in 2019 on the then-draft Social Card Law and submitted further comments in 2021. However, the Commissioner has not had a role in monitoring the implementation of the Social Card registry since then and has not been invited to provide any assessment or comment following implementation. The Commissioner’s office noted several concerns about whether data protection standards were met, especially around the necessity and proportionality of data being processed – one of the conditions necessary for any legitimate interference with an individual’s right to privacy – and that some of the provisions of the Law were too vague.172 On the question of proportionality, the Commissioner found that no valid explanation was given for why the scale of data collection was needed, concluding that “there is no explanation [for] why it is necessary and why this is done, so how can it be proven to be proportional”.174 

The Ministry for Labour is the authority ultimately responsible for ensuring data protection safeguards and standards. However, the Commissioner was unaware of which specific safeguards the Ministry had put in place, raising concerns that the Social Card registry lacks adequate safeguards and, therefore, poses a risk to the right to privacy. 

The CoE has developed guidance for member states on how to stop Artificial Intelligence (AI) from impeding human rights, particularly the right to remedy. Although the document refers specifically to AI, many of its recommendations are relevant to any system using automation. This is because similar risks and concerns exist about using both AI and automated systems in public sector decision-making, and the guidance touches on automation (albeit in the context of AI). The CoE recommends that states regularly test, evaluate, report and audit systems using automation both before, during, and after implementation to ensure that human rights and data protection standards are met and that no undue discrimination is caused at any stage of using the system.175

A declaration by the CoE Committee of Ministers on the risks of decision-making assisted by computers or enabled by AI in the field of social assistance outlines that these systems, if “not developed and used in accordance with principles of transparency and legal certainty, amplify bias and increase risks”.176 The declaration also establishes the need for states to ensure that these systems are developed and implemented in accordance with the principles of ”legal certainty, legality, data quality, non-discrimination and transparency” and that robust human oversight and monitoring needs to be in place.177

The UN Special Rapporteur on extreme poverty and human rights recommends “inclusive mechanisms” in data and monitoring processes that account for the “asymmetries of power” between rights holders and duty bearers.178 Without these inclusive mechanisms, those who are least likely to enjoy their rights will be least able to participate in assessing these policies. This will, in turn, lead to policies being less reflective of marginalized people’s needs and concerns, potentially leading to further human rights violations and discrimination. 

Footnotes

[169] UN Special Rapporteur on extreme poverty and human rights, Report: Digital technology, social protection and human rights, 1 October 2019, UN Doc. A/74/93, p. 5. 

[170] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International

[171] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International

[172] Equalities Commission, Opinion on the Draft Social Card law, 11 October 2019, number 011-00-00032/2019-2, Commissioner for information of public interest and protection of personal data, Opinion on the Draft Social Card law, 15 December 2020, number 073-12-2598/2020-02. See also Amnesty International, Serbia: Social Card law could harm marginalized members of society – legal opinion, 28 November 2022, https://www.amnesty.org/en/latest/news/2022/11/serbia-social-card-law-could-harm-marginalized-members-of-society-legal-opinion

[173] Interview with the Office of the Data Protection Commissioner, 28 April 2023, Belgrade, Serbia.

[174]  Interview with the Office of the Data Protection Commissioner, 28 April 2023, Belgrade, Serbia.

[175] CoE Committee of Ministers, Recommendation to member States on the human rights impacts of algorithmic systems, 1 April 2020, CM/Rec (2020)1, (d) “immediate rectification” of “any significant restrictions on human rights that are identified during the testing of such systems”.

[176] CoE Committee of Ministers, Declaration by the Committee of Ministers on the risks of computer-assisted or artificial-intelligence-enabled decision making in the field of the social safety net, 17 March 2021, COE doc Decl(17/03/2021)2.

[177] CoE Committee of Ministers, Declaration by the Committee of Ministers on the risks of computer-assisted or artificial-intelligence-enabled decision making in the field of the social safety net, 17 March 2021, COE doc Decl(17/03/2021)2.

[178]  UN Human Rights Council (HRC), Report of the Special Rapporteur on extreme poverty and human rights, Magdalena Sepúlveda Carmona, 22 May 2014, A/HRC/23/36, para. 22. 

an illustration of a person tangled and wrapped in long lines of paperwork

Chapter 9 – Lack of transparency

The Serbian Ministry for Labour has so far not been fully transparent with the general public nor with Amnesty International about the operation of the Social Card registry. This lack of transparency makes any analysis of how the registry operates and the automation within it difficult. It also undermines Serbia’s commitment to the principles of transparency in public affairs. Transparency about the Social Card registry implementation would allow affected communities and broader civil society to monitor its operation, identify concerns in a timely manner, and evaluate the system’s efficacy.179 Greater transparency about the use of automation in the Social Card registry would also help build trust among its users and wider civil society.

The Ministry of Labour has cited concerns over intellectual property rights as a reason to refuse Amnesty International’s requests to access information about the Social Card registry and to deny Freedom of Information requests filed by civil society organizations.180 Intellectual property rights or trade secrets must not take primacy over meaningful transparency over the human rights impacts of automated systems which affect people’s lives and livelihoods. Furthermore, the Ministry argued that providing the source code for the registry would expose the personal data of people in the registry to danger.181 However, this is not the case. Some states, including the Netherlands and France, have chosen to make the source codes for some systems using digital technologies public and, therefore, open them to scrutiny. The source code information does not include personal data stored in the registry, and its release would not jeopardize privacy or personal data. 

While Amnesty International calls for transparency in matters related to Serbia’s Social Card registry, transparency in itself is not the ultimate goal. Instead, transparency is a mechanism for accountability and reducing the current burden of gathering information for civil society and impacted communities to prove adverse impact. If a system is causing or can cause human rights harms, then transparency is not a sufficient mitigation measure to justify its deployment.

Footnotes

[179] UN HRC, The right to privacy in the digital age, 4 August 2022, UN Doc. A/HRC/51/17, p. 15.

[180] Meeting with representatives of Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 28 April 2023, notes on file with Amnesty International; Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[181] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International.

an illustration of a group of people walking into a grey, dark maze

Chapter 10 – The role of the World Bank

Governments worldwide have turned to digital technology, including automation, to facilitate public service delivery, including social security. Such solutions are often promoted as measures to ensure a fairer and more efficient distribution of often limited state resources and reduce costs. 

As one of the largest providers of social protection loans globally, the World Bank has played a crucial role in advocating for greater automation in social protection systems, particularly in low and middle-income countries. A key aspect of this is more effective poverty targeting, a process of prioritizing cash transfers and other benefits among individuals and groups based on their socio-economic status. 

Although widely used in low and middle-income countries, poverty targeting as a concept has faced significant criticism. When they rely on narrow indicators to measure poverty, such programmes are prone to errors that can exclude people who need social assistance and are generally inadequate for protecting people against economic insecurity.182 

The risks of poverty-targeting are further exacerbated by the increasing integration of digital and automated technologies to determine people’s eligibility for social security programmes. A central element of poverty targeting digitalization is the creation of social registries, or information systems that enable data collection about recipients and potentially an assessment and determination of their eligibility for some form of social protection. In some cases, social registries rely on static information, such as data about households collected through a census and household surveys (static social registry). Others are integrated with other government institutions and automatically receive regular, up-to-date data from existing registries (a dynamic social registry). A guidance note from the World Bank on social registries states that a critical function of social registries is the “automation of the processes for assessing needs and conditions to determine potential eligibility for social programmes”. 183

Social registries – and the automation of poverty targeting in general, have faced intense scrutiny. Social policy organization Development Pathways found that relying on often out-of-date and inaccurate data can lead to many errors and exclusions, some of which are also documented in this report. They also argue that social registries are de facto “systems of exclusion, not inclusion” because they often exclude most of society’s poorest members from social protection. A growing body of evidence also suggests that automated poverty-targeting can be prone to discriminatory outcomes.316 

The World Bank has itself acknowledged the risks of social registries, however, it has maintained that effective poverty-targeting remains the best available way to reach people in poverty in most countries with limited resources. In the context of growing concerns about the human rights impacts of using technology in national welfare systems, it is essential that the World Bank conducts – and encourages governments to conduct – robust human rights risk assessments while designing and implementing such programmes to recognize and mitigate potential harmful impacts.  

According to the Tilberg Guiding Principles on World Bank, IMF and Human Rights, as an independent specialized international organization under the UN, the World Bank has international legal obligations to “take full responsibility for human rights respect in situations where the institutions’ own projects, policies or programmes negatively impact or undermine the enjoyment of human rights”. The International Financial Corporation’s Sustainability Framework and Corporate Governance provides international benchmark practices for assessing projects’ environmental and social risks. However, there is an urgent need to simultaneously put in place human rights due diligence exercises to ensure that the World Bank’s support does not result in human rights violations. 

In 2013, the former UN Special Rapporteur on the right to adequate housing urged the World Bank to adopt safeguard policies, align with the international human rights obligations of its member states and incorporate the UN Guiding Principles on Business and Human Rights into its operations. Due to its ubiquitous presence and influential role in providing national-level financial and technical assistance to key reforms in low and middle-income countries, the World Bank is also in a strong position to support the governments it works with to meet their human rights obligations. 

Footnotes

[182] Amnesty International’s research in Lebanon on the World Bank funded Emergency Crisis and Covid-19 Response Social Safety Net Project (ESSN) found that this system excluded many people in poverty from social assistance due to narrow criteria and a narrow registration period.

[183] A critical ‘back-office’ function of Social Registries is the automation of the processes for assessing needs and conditions to determine potential eligibility for social programs.[…] This information is then transformed into measures of welfare (such as means-testing, hybrid means testing, proxy means test scores, multi-dimensional poverty scores, etc.) and then compared to pre-defined eligibility criteria for user program(s) to determine potential eligibility. This process is automated through the software applications component of the Social Registry that supports the business rules and logic to carry out these algorithms.” World Bank Group, Social Registries for Social Assistance and Beyond: A Guidance Note and Assessment Tool, July 2017, https://documents1.worldbank.org/curated/ar/698441502095248081/pdf/117971-REVISED-PUBLIC-Discussion-paper-1704.pdf p12-13 

[184] Human Rights Watch, “Automated Neglect: How The World Bank’s Push to Allocate Cash Assistance Using Algorithms Threatens Rights”, 13 June 2023, https://www.hrw.org/report/2023/06/13/automated-neglect/how-world-banks-push-allocate-cash-assistance-using-algorithms;  Amnesty International, Actions Speak Louder Than Words: The World Bank Must Promote Universal Social Protection (Index: POL 40/7224/2023), 10 October 2023, https://www.amnesty.org/en/documents/pol40/7224/2023/en


The World Bank and Serbia’s social card system

The World Bank funded the establishment of the Social Card as part of an 82,600,000-euro loan to the Serbian government to support the “Public Sector Efficiency and Green Recovery Development Policy”. The adoption of the Social Card Law was a condition of the Bank approving the loan.185 Developing social welfare registries is a key pillar of the World Bank’s strategy for Western Balkan countries, which seeks to improve poverty targeting in their social protection systems and “make [such systems] fairer, more sustainable, more effective and better able to respond to people’s needs”. According to the World Bank, the Social Card Law “will enable better determination of the socio-economic status of applicants and their rights to benefits; increase the efficiency of and transparency of the social assistance system; and enable the government to improve its shock response”. 

The World Bank considered the development of a Social Card registry as “essential to identify beneficiaries quickly during crises and expand coverage of social assistance support”.186 It said that an integrated registry would allow authorities to better target poverty, that is, to effectively identify households in need based on data rather than static demographic characteristics that could be out of date or inaccurate. 

In a written response to Amnesty International, Serbia’s Ministry of Labour stated that the authorities have cooperated with the World Bank since 2017 and were “helped to a very significant extent professionally” in the process of digitizing the new social protection system.187 The World Bank’s assistance included a loan to set up the registry and support through multiple stages of the project to create the Social Card registry – from conceptualisation to the drafting of the Social Card Law – and finally to its implementation.188

The World Bank has not responded to multiple requests during this research in which Amnesty International sought information about how the World Bank ensured that social registries whose development it supported safeguarded human rights and if it carried out any human rights impact assessments prior to the development of the Social Card registry in Serbia. It is therefore not clear whether the Word Bank has conducted human rights due diligence to determine whether the system it funded had a discriminatory impact on the rights of marginalized groups and to ensure that the technology used met human rights standards.

Footnotes

[185] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International. 

[186] Official correspondence from Ministry of Labour, Employment Veterans and Social Affairs, 24 November 2023, on file with Amnesty International. 

[187] Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International.

[188] “The support related to numerous aspects, the main of which are: preparation of the conceptual project of the Social Card register, analysis of information systems in social protection and proposal for improvement,  preparation of technical specifications for the Social Card in the 1st and 2nd phases, analysis of the legal framework in R. Serbia in connection with the drafting of the Law on the Social Card, preparation of the working version and drafting of the Law on the Social Card, support for the implementation of the Social Card register.” Official correspondence from Serbia’s Ministry of Labour, Employment Veterans and Social Affairs, 30 August 2023, on file with Amnesty International. 

Chapter 11 – Conclusion and Recommendations

The introduction of technology into the public sector, and in particular in social protection systems, is often portrayed as a noble enterprise designed to improve people’s access to government services and increase efficiency. Yet, as the example of the Social Card registry in Serbia shows, introducing automation into an already inadequate social security landscape without first addressing existing flaws and structural discrimination will inevitably exacerbate and entrench these problems and further undermine people’s right to social security. In little over a year, the Social Card registry in Serbia resulted in possibly thousands of the most socially and economically vulnerable people losing the social assistance that they relied on. Without proper safeguards and protections in place, people from marginalized communities, such as Roma and people with disabilities, were disproportionally affected. 

Amnesty International considers that any introduction of technology, including low-level automation and semi-automated decision making, as is used in the Social Card registry, must be carefully considered and weighed against the potential risks. It is crucial that the introduction of any technology not only be accompanied by adequate and robust human rights impact assessments throughout the lifecycle of the system, from design to deployment, and effective mitigation measures, but also that communities who will be impacted by the system be consulted and that any changes are communicated in a clear and accessible way. International financial institutions, such as the World Bank, should take their responsibilities seriously and ensure that any funding or technical support provided to programmes that aim to introduce technology into social protection do not infringe on human rights. If a system is found to have the potential to harm human rights and that harm cannot be effectively prevented, it must not be deployed. 


Recommendations to the Serbian Government

Amnesty International calls on the Serbian government and relevant Ministries to act in line with their human rights responsibilities and to: 

  • Adopt concrete policies and measures on poverty reduction, focusing particularly on people who face systemic and inter-generational poverty, such as Roma and people with disabilities, and provide sufficient financial resources to ensure that such measures reach people living in poverty.   
  • Ensure that social assistance is in line with Serbia’s human rights obligations to the right to social security, including adequate payments, equal access, and available support for the duration of time that people need it. Specifically, 
  • Increase the level of budget allocations for means-tested programmes, including financial social assistance and child benefits, designated to aid those living in extreme poverty; 
  • Urgently increase the amount of social assistance support provided to ensure that it is sufficient to guarantee the right to an adequate standard of living, as per ECSR’s 2022 Conclusions; 
  • Remove all restrictive and discriminatory eligibility criteria that prevent people living in extreme poverty from accessing social assistance and raise the minimal income threshold to qualify for social assistance;  
  • Consider allowing people living in extreme poverty to combine modest income with social assistance to ensure that they can meet their basic needs. 
  • Conduct an independent human rights impact assessment of the Social Card Law and its application. This impact assessment needs to include, at the very minimum, an evaluation of the Social Card registry’s potentially discriminatory effects on specific groups such as Roma or persons with disabilities. 
  • Conduct an assessment of how the Social Card registry system has worked since its implementation to determine whether its semi-automated decision-making, data processing and appeals process are in line with international human rights standards and international and domestic data protection frameworks. 
  • Assess the role of social workers to determine whether they fully understand how the Social Card system works and are able to maintain meaningful control over the system throughout the social assistance application process. 
  • Provide social workers with additional training and capacity building where necessary to address and prevent issues such as automation bias. 
  • Make public information about how the Social Card registry works, including the Data Processing Warehouse, to ensure full transparency and compliance with the right of access to information. 
  • Conduct comprehensive public outreach campaign that targets communities most likely to be in receipt of social assistance and other benefits to ensure that they understand how the new system works; how it affects the eligibility requirements for social assistance and other benefits; the importance of good record-keeping in other citizens databases; and circumstances in which social assistance could be removed or reduced. An information and public outreach campaign should be made in accessible language and promoted by means that are most available to people belonging to marginalized communities. 
  • Conduct public consultations with key stakeholders, including affected communities, on the Social Card registry to facilitate a proper understanding of its operation, function and potential impact and receive feedback from existing users. 
  • Ensure that social assistance applicants receive clear and accessible information about how decisions are made in their cases, how to appeal such decisions, and, where needed, ensure that applicants receive support in lodging their appeal, including legal or financial support. 
  • Ensure that applicants affected by the Social Card registry are given access to an effective remedy, including prompt and adequate reparation for any harm suffered in the process. 
  • Put in place independent monitoring and oversight of the Social Card registry. 
  • Ensure that policy making reflects the needs of marginalized communities in order to prevent discrimination, including discrimination in effect. This should include providing adequate support to enable marginalized groups such as Roma and people with disabilities to access their right to social security. 
  • Ensure adequate data quality standards such that the registry can be used effectively by social workers, in compliance with IHRL and to enable greater inclusion in Serbia’s welfare system. This includes: 
  • Improving existing data collection mechanisms such that they contribute to accurate and current understandings of beneficiary applicants – while ensuring the right to privacy, alignment with domestic data protection frameworks and adherence to established best practices on processing demographic data. 
  • Implementing inclusive data collection practices such that marginalized groups can be represented accurately in the social registry. 
  • Where new data collection mechanisms must be added to meet adequate quality standards, ensuring these comply with the principle of data minimization, IHRL and domestic data protection frameworks”. 
  • Amend national anti-discrimination legislation to recognize intersectional discrimination as well as the denial of reasonable accommodation as a specific form of disability-based discrimination, as per CRPD recommendations. 
  • Allow for a greater role for the Commissioner for Information of Public Importance 
    and Personal Data Protection and the Commissioner for the Protection of Equality in the oversight and monitoring of the Social Card registry, and more broadly any uses of automated or semi-automated systems in the public sector that have the potential to impact human rights.

Recommendations to the World Bank

  • Amnesty International calls on the World Bank to act in line with its human rights responsibilities and its commitment to universal social protection, and ensure that its funding and technical support of social protection programmes and that the introduction of potentially rights infringing technologies should not be a precondition of funding. 
  • Phase out support for narrow poverty targeting programmes, and replace these with programmes designed to progressively achieve universal social protection. 
  • Conduct and make public an independent assessment of the human rights impact of introducing Serbia’s Social Card registry and provide suggested remedial measures in the instance of any human rights risks. 
  • Align with ILO General Recommendation 202 calling for the establishment of Social Protection Floors, which, at a minimum, ensure that everyone has access to adequate levels of healthcare, maternity and paternity benefits, child benefits, basic income security for those unable to earn a sufficient income in cases of sickness, unemployment, and disability, and basic income security for older persons. 
  • Meet a level of adequacy that allows people to access their right to an adequate standard of living. 
  • Factor in and address the multiple and intersectional forms of discrimination that many groups face when claiming their human rights, including women, people with disabilities, older people, people living in poverty, people working in the informal sector, children and people belonging to racialized and otherwise minoritized communities. 
  • Guarantee transparency and accountability through accessible grievance redress and appeals processes, provide data protection through secure systems, and ensure that any automated decision-making used is in line with human rights standards on privacy, equality and non-discrimination as well as with regional and domestic data protection frameworks and that it is never used in a way that could lead to discriminatory or harmful outcomes. 
  • Include an independent human rights impact assessment of any social protection system and any digital technologies that underpin it, as well as follow-up assessments to evaluate its application and determine any potential issues after its introduction, including potentially discriminatory effects on specific groups. 
  • Ensure that when a new system is introduced, information about how it functions, its criteria and any appeals mechanisms in place to challenge decision-making, are widely disseminated in an accessible format. 
  • Ensure that any new systems introduced comply with state-of-the-art requirements on data and data governance, documentation and recording keeping, transparency and provision of information to users, human oversight, robustness, accuracy and security, as well as relevant digital and human rights standards. 
  • Ensure that companies providing social security systems comply with the UN Guiding Principles on Business and Human Rights, as well as relevant regional and national corporate sustainability and due diligence frameworks.

Recommendations to all states

  • Ensure full transparency about automated or semi-automated systems used by a public authority or on their behalf. This includes providing information on the system’s existence and operation, what data is used, which databases are merged, how databases are merged, how personal data is processed, and details about the decision-making process, including the degree of human involvement and interaction with the automated system. 
  • Ensure that automation is used in line with human rights standards, including on privacy, equality and non-discrimination as well as data protection standards, and that it is never used in a way that could lead to people being discriminated against or otherwise harmed. 
  • Ensure that rights holders are informed when automation or semi-automation is used to process their data. The information provided should be concise, easily understandable, and accessible, including for persons with disabilities, people who are not digitally literate, and marginalized communities, and this information should include details about the system’s purpose or task. Furthermore, rights holders should be told how to access more information on the system if they require it (namely, legislative framework), as well as information on the rights that may be impacted, including the right to social security or equality and the details on any appeals or complaints mechanism available to the individual. 
  • Implement a mandatory and binding human rights impact assessment of any public sector use of automated and algorithmic decision-making systems. This impact assessment must be carried out during the system design, development, use and evaluation and – if relevant – retirement phases of automated or algorithmic decision-making systems. The impact on all relevant human rights, including social and economic rights, must be assessed and properly addressed in the human rights impact assessment. The process should involve relevant stakeholders, including independent human rights experts, individuals from potentially impacted, marginalized and/or disadvantaged communities, oversight bodies, and technical experts. 
  • Establish comprehensive and independent human rights oversight mechanisms over the use of automated or semi-automated decision-making systems, including over civil and political rights and social and economic rights, to strengthen accountability mechanisms and increase human rights protection. An oversight body should be granted the mandate, powers, expertise and capacity to oversee human rights protection in the use of automated or semi-automated public sector decision-making systems, issue guidance, and hold designers, developers and users to account in binding decisions, as well as raise public awareness about relevant rights and support available to potentially impacted people. 
  • Provide meaningful accountability, effective remedy and redress for human rights harms linked to the use of automated or semi-automated decision-making systems. This should include, for example, creating clear, independent and accessible processes for redress and designating public sector roles to be responsible for the timely remedy of human rights harms, subject to accessible and effective appeal and judicial review. 
  • Ensure that impacted individuals have the right to information about the use and functioning of the automated or semi-automated decision-making system; the right to lodge a complaint with the relevant oversight body; allow representation of natural persons and the right for public interest organizations to lodge a complaint with the oversight body; effective remedy against the oversight body in case of violation of rights. 
an illustration of two children playing with dolls between a group of houses.