According to Dutch News, The Hague District Court held that the system is in conflict with EU human rights and privacy rules.
The so-called Dubbed System Risk Indication (SyRI), an automatic tool machine learning was used by local authorities to track people who were suspected of committing social security fraud.
SyRI creates a 'risk profile' for people who have previously committed a social security fraud and then scans for 'similar' citizen profiles, thereby creating a list of all possible profiles of people who could potentially be involved in a similar fraud. .
The data collection "was targeted exclusively at low-income and minority neighborhoods," according to UN human rights spokesman Philip Alston. In essence, the tool targeted people from economically weaker neighborhoods, with no other incriminating element in its possession.
The Netherlands State may appeal the decision.
The Human Rights Watch considers the ruling a victory and said the mandate "has set an important precedent for protecting the rights of the financially weaker in the era of automation".
“Governments that rely on data analytics to police access to social security - like those in USAThe United Kingdom and Australia - should heed the court's warning about the human rights risks involved in treating social security beneficiaries as permanent suspects, ”the civil rights group added.