Study: "Dirty data" taints predictive policing software
Marie Donlon | April 12, 2019
Researchers from New York University (NYU) School of Law and NYU's AI Now Institute are suggesting that predictive policing systems may intensify discrimination in the criminal justice system by relying on so-called “dirty data,” according to a new study.
Because law enforcement has been heavily scrutinized in recent years for allegedly inordinate aggression toward minority suspects, experts have wondered whether technologies like predictive policing software — software used to help law enforcement identify potential criminal activity and its perpetrators based on data analytics — could potentially diminish instances of discrimination. However, NYU researchers concluded that such techniques potentially worsen discrimination when informed by dirty data that emerges from flawed or racially biased practices.
To reach that conclusion, researchers looked at data from case studies in Chicago, New Orleans and Arizona’s Maricopa County.
"We chose these sites because we found an overlap between extensively documented evidence of corrupt or unlawful police practices and significant interest, development, and current or prior use of predictive policing systems. This led us to examine the risks that one would influence the other," explained Jason Schultz, a professor of clinical law and one of the paper's co-authors.
The team pinpointed 13 jurisdictions (including those previously mentioned) with recorded instances of biased and unlawful police practices that simultaneously explored or deployed predictive policing techniques. For instance, researchers noted that the Chicago Police Department incorporated a system for identifying those at risk of either becoming a victim or a perpetrator of a homicide or a shooting while at the same time the department was under federal investigation for unlawful police practices. According to the study, the very same demographic of residents identified as targets of Chicago’s policing bias by the Department of Justice were essentially the same as those identified by Chicago’s predictive policing systems.
"In jurisdictions that have well-established histories of corrupt police practices, there is a substantial risk that data generated from such practices could corrupt predictive computational systems. In such circumstances, robust public oversight and accountability are essential," Schultz said.
The study, "Dirty Data, Bad Predictions: How Civil Rights Violations Impact Police Data, Predictive Policing Systems, and Justice," appears on SSRN.
But, hoo-boy, if you were to accuse scientists of using 'dirty data' in the global warming debate, you are going to shouted down as a denier.
I'm all for law and order, but the term "predictive policing" sounds dangerous to me, especially if the data has to be "cleaned".
It's not surprising - bias in, bias out. Dirty data indeed.
In reply to #3
Is it really bias or perceived bias? There seems to be a lot of confirmation bias going on.
There also seems to be no shortage of those who wish to ascribe an institutional reason for individuals' behaviors.
In reply to #4
Bottom line, an AI or other system of prediction is only as good as the data. A system that is fed biased input is going to have a biased output.
A program, machine or AI is only as good as its handlers. I don't need a confirmation bias for this conclusion to be valid.
AI, Actual Idiots is what it means. And they are correct, as we allow this to be forced on us without any input or rejection.
I think this was trialed on Julian Assange this week.
In reply to #6
AI, I think Artificial Idiots might capture it.
In reply to #7
Serious question: If I shoot and kill and Ai machine am I a murderer? If I switch it off is this humane or am I a murderer, (stone cold heartless killer)?
In reply to #8
I am entirely in favor of a firm hand with the machines. Call me heartless.
Seriously, the RCMP tried to charge a man with attempted murder here a couple years ago, for firing on one of their robots. You see, they believed the man was mentally ill, and so instead of whatever they do with normal people, they sent this machine into his house to
harrass himdeliver their message. Because that wouldn't freak you out, if you were having an 'episode'.In reply to #9
They are pulling out plonkers, next we will be burying our power tools, with a full service and some professional wailers. Rust in peace.
It does not say much for the RCMP. They should go back to horses, they were far more clever.
Just how much more stupid, ridiculous, pathetic, beyond any, never mind human, comprehension can humans get. (Not a rhetorical question). Please confirm, this is planet earth? I think aliens have abducted planet earth. Mork calling Ausin, come in please.
In reply to #10