They Secretly Scored Half a Million People for Future Crimes. Including Children. And When It Failed, They Said Nothing.
- Michael Routhier
- Jun 29
- 6 min read
What if someone, right now, today, had already decided whether you were likely to commit a crime?
Not based on anything you did. Based on where you live. Based on whether your family needed help paying for school meals. Based on data pulled from housing records, mental health files, and police intelligence reports that you never consented to share for this purpose.
What if that decision was made about you years ago, and you never knew?
What if it was made about your child?
That is not a hypothetical. That is Bristol, England. Right now. In 2026. In a democracy.
Let's start at the beginning. Because you need to know exactly what was built.
A decade ago, Avon and Somerset Police and Bristol City Council built something called the Think Family Database. Over time, it grew to store information on nearly half a million people. Half. A million. That is roughly half the entire population of Bristol.
What kind of information? Police intelligence reports. Housing status. Mental health records. Teenage pregnancies. Whether a family enrolled in a parenting course. Whether a child received free school meals.
And then, they fed that data into algorithms. Algorithms designed to predict which children were at risk of becoming criminals. Which people would commit burglaries. Who would go missing. Who would become a domestic violence victim.
Now I want you to sit with that for a moment. Because I have a question.
When you hear "free school meals" as a data point in a crime prediction algorithm, what does that tell you about who this system was actually targeting?
It tells you everything.
Free school meals are a poverty marker. They are provided to families who cannot afford to feed their children during the school day. They are not a criminal indicator. They are not a risk factor. They are evidence of economic hardship, the kind of hardship that has always, throughout history, been used to justify surveillance of the poor while the wealthy operate without scrutiny.
Let me ask you directly; do you think the algorithm was scoring the children of Bristol's wealthiest families?
Do you think the data included the financial records, the private school enrollment, the country home addresses of the people who built this system?
Of course it didn't. It never does.
This is the oldest story in human civilization. The powerful build systems of control, and they aim them downward. Always downward. Never at themselves. Never at the boardrooms. Never at the people whose decisions actually shape the conditions that produce poverty, desperation, and crime in the first place.
The algorithm didn't create that injustice. It industrialized it.
And here is the part that should make your blood run cold.
WIRED obtained performance data covering more than 36,000 model performance scores from the British police system. Independent experts reviewed them. Their conclusion? In some cases, the results showed "genuinely poor predictive performance".
The council quietly scrapped the algorithms. Concluded they were "not fit for operational use".
No public announcement. No accountability. No explanation to the families whose children had been scored. No letters sent. No records corrected.
And the man who asked, plainly; "Is that incident still marking me?", nobody could answer him. Nobody was required to.
So I want to ask you, Socrates-style; if a system was used to make decisions about half a million people, and that system was later found to be untrustworthy, what is the moral obligation of the people who ran it?
What does justice require?
And if they say nothing, do nothing, and correct nothing, what do we call that?
I'll tell you what we call it.
The ancient Stoic philosopher Musonius Rufus, a man who lived 2,000 years ago and still understood power better than most of our institutions do today, was direct; "Evil is injustice and cruelty and indifference to a neighbour's trouble."
Indifference. Not ignorance. Indifference.
Bristol City Council knew. They scrapped the system. And they said nothing to the half million people whose data built it.
That is not an administrative oversight. That is a moral choice. And Musonius Rufus named it correctly.
Now I want to connect this to something I've talked about on this platform before.
In my post on The Dictator's Algorithm, I documented how authoritarian governments; China, Russia, Iran, are building AI surveillance states to monitor, score, and control their populations.
And I want to ask you a question that I genuinely want you to think about before you answer;
What is the difference between a Chinese AI system that scores citizens for social compliance, and a British police algorithm that scores children for future criminality based on whether their parents needed help feeding them?
The geography is different. The stated intention is different. The political system is different.
But the architecture is identical. A population surveilled without meaningful consent. Scores assigned without transparency. Decisions made that affect lives. No mechanism for challenge. No requirement for accountability.
The Dictator's Algorithm doesn't only live in Beijing. It lives in Bristol. And if it lives in Bristol, you need to ask yourself, where else does it live?
What the European Union needs to do. Right now.
The EU AI Act, which came into force in stages beginning in 2024, specifically prohibits "real-time remote biometric identification systems" and places high-risk classification on AI used in law enforcement.
But here is my question to the European institutions, to the regulators, to the human rights bodies that exist precisely for moments like this;
If a system that secretly scored half a million people, including children, for future criminality was found to be unreliable, was quietly scrapped, and left those half million people with no notification and no recourse; does that meet the definition of high-risk AI under the EU AI Act?
And if it does, why has there been no formal investigation?
The EU has consistently positioned itself as the global leader on AI regulation. Prove it. Not with white papers. Not with framework documents. With an actual accounting of what was built in Bristol, who authorized it, who was harmed by it, and what is owed to them.
Because I'll ask the question Socrates would ask; what is the point of a law that protects people on paper but not in practice?
Let me come back to the child.
Somewhere in Bristol right now, there is a person who was flagged as a child, their family's poverty used as a data point, their future pre-decided by an algorithm that experts have since found to be unreliable.
That person is living their life. Going to work. Perhaps raising their own children. And somewhere in a database that the council will not discuss, in a record they may never be permitted to see, a score exists.
Marcus Aurelius wrote; "We were born to work together. To obstruct each other is unnatural."
A system that decides a child's future before that child has lived it is not working with that child. It is working against them. It is obstruction, built with taxpayer money, run by public institutions, aimed at the people least equipped to challenge it.
And when it was found to be broken, silence.
Here is my final question. And I want you to answer it, not to me, but to yourself.
If this had been done to the children of Bristol's wealthiest families, if the algorithm had flagged the sons and daughters of politicians, executives, and police commissioners for future criminality based on their parents' financial records:
Would the council have scrapped it quietly?
Or would there have been a reckoning?
You already know the answer.
And that answer is exactly what The Virtuous Machine exists to say out loud.
"It is a denial of justice not to stretch out a helping hand to the fallen; that is the common right of humanity." - Seneca
Half a million people were scored. Many of them were children. Many of them were poor. The system failed. The institutions said nothing.
We are saying something.
All claims in this article are grounded in WIRED's two-year investigation, Liberty Human Rights documentation, and publicly available performance data obtained through disclosure requests. Every factual assertion is independently verifiable. This is not opinion dressed as evidence. This is evidence, with the moral weight it deserves.
➡️ Want to understand how authoritarian governments are already using AI to control and surveil their populations? [Read: The Dictator's Algorithm - How Authoritarian Governments Are Quietly Shaping the AI That Shapes You].
➡️ Concerned about how AI is being deployed against ordinary people with zero accountability? [Read: They Asked the AI Nicely. It Handed Over the Keys - The Meta Instagram Hack and What It Means for Your Bank Account].
➡️ Come talk about this in the community - no jargon, no judgment. [Join us free at Tech 4 Grown-Ups].
Source:
WIRED / Liberty Investigates - British Police Built a Sprawling Crime-Prediction Machine. Some Results Couldn't Be Trusted.
Additional Sources:
The Guardian - UK use of predictive policing is racist and should be banned, says Amnesty.
Liberty Human Rights - Liberty report exposes police forces' use of discriminatory data to predict crime.
Comments