Courts across America are using algorithmic risk assessment tools to make decisions about bail, sentencing, and parole. These systems promise objectivity and efficiency, but they often deliver something far more troubling: the automation of existing biases wrapped in a veneer of scientific authority.
The Promise and the Reality
The appeal of algorithmic decision-making in criminal justice is understandable. Human judges are demonstrably inconsistent, influenced by factors as arbitrary as whether they have eaten lunch. If software could apply consistent, evidence-based criteria to predict recidivism and flight risk, it could reduce both bias and unnecessary incarceration.
The reality falls dramatically short of this promise. Risk assessment algorithms are trained on historical criminal justice data, which reflects decades of racially disparate policing, prosecution, and sentencing. When you train a prediction model on biased data, you get biased predictions. The algorithm does not eliminate racism; it launders it through mathematics.
The COMPAS Problem
The most widely studied risk assessment tool, COMPAS, was found by ProPublica to be nearly twice as likely to falsely flag Black defendants as future criminals compared to white defendants. The tool’s developers contested the methodology, but the fundamental critique remains valid. Any system trained on data from a discriminatory system will reproduce and potentially amplify that discrimination.
The Black Box Dilemma
Many of these algorithms are proprietary, meaning defendants cannot examine the tools being used to determine their freedom. This represents a fundamental violation of due process. When a judge makes a sentencing decision, the reasoning is articulated in open court and subject to appellate review. When an algorithm contributes to that decision, its logic is hidden behind trade secrets and corporate intellectual property claims.
The Sixth Amendment guarantees the right to confront witnesses and evidence. An algorithm that cannot be examined, challenged, or understood by the defense is incompatible with this constitutional principle, regardless of how accurate its developers claim it to be.
What Should Be Done
This does not mean technology has no role in criminal justice. But its deployment must be governed by principles that prioritize fairness and accountability over efficiency. Any algorithm used in judicial proceedings should be open-source and subject to independent audit. Defendants must have the right to challenge algorithmic assessments with the same vigor they challenge human testimony.
Validation studies should be mandatory, with algorithms tested for disparate impact across racial, gender, and socioeconomic categories before deployment. Tools that demonstrate significant bias should be barred from judicial use until the bias is corrected and independently verified.
Most importantly, algorithmic assessments should inform rather than determine judicial decisions. A risk score presented as one factor among many is fundamentally different from a risk score that effectively dictates outcomes. Judges must retain genuine discretion, and that discretion must be exercised with full understanding of an algorithm’s limitations.
The criminal justice system is already plagued by inequality. Introducing opaque, unaccountable technology into this system without rigorous safeguards does not represent progress. It represents the industrialization of injustice.
