# Criminal Risk Assessment Instruments Risk assessment instruments score a defendant's probability of failing to appear or being rearrested. They are used at bail, sentencing, and parole across most American jurisdictions. COMPAS, developed by Northpointe (later Equivant), is the best-documented; the Public Safety Assessment, the LSI-R, and numerous state-built tools are also in wide use. The instruments predict from criminal history, age at first offense, employment and residential stability, substance use, and — in some tools — questionnaire responses about attitudes and family background. **Race is not an input to any mainstream instrument.** ## The reform case, stated fairly These tools were substantially a *progressive* project, and that is essential context that gets lost. The alternative is judicial discretion in bail-setting, which produced enormous unexplained variation between judges, systematic racial disparity, and a cash-bail system that jailed poor defendants pretrial while releasing wealthy ones facing identical charges. Risk assessment was pitched — by bail-reform advocates, not by prosecutors — as the instrument that would let jurisdictions release more people more safely by replacing intuition with evidence. Several jurisdictions did reduce pretrial detention after adoption. **Whoever is right about the fairness question, the honest comparison is not against a neutral baseline. It is against cash bail and unstructured discretion, both of which performed badly.** ## The 2016 dispute ProPublica's "Machine Bias" investigation examined COMPAS scores in Broward County and reported that among defendants who did *not* go on to reoffend, Black defendants were roughly twice as likely as white defendants to have been labeled high-risk — a false-positive disparity. Northpointe responded that COMPAS was **calibrated**: among defendants assigned any given risk score, the actual reoffense rate was approximately equal across racial groups. A score of 7 meant the same thing regardless of race. Both analyses were correct. They were measuring different things, and this is the crux: neither party had made an arithmetic error, and the disagreement could not be resolved by better data or a better model. ## The impossibility result Within a year, two independent groups established why. Alexandra Chouldechova (2017), and Jon Kleinberg, Sendhil Mullainathan and Manish Raghavan (2016), proved that **calibration within groups and equal false-positive and false-negative rates across groups cannot all hold simultaneously whenever base rates differ between groups — unless the prediction is perfect.** Since arrest rates differed between the groups in question, and no recidivism predictor is remotely perfect, the conditions for the impossibility applied directly. COMPAS could satisfy Northpointe's fairness criterion or ProPublica's, not both. See [[Fairness Impossibility Results]] for the general statement. **This transformed the debate.** The question stopped being "is this algorithm biased?" and became "which fairness criterion should this system satisfy, and who decides?" — a question of values that no amount of technical work can answer. ## The problems the impossibility result does not cover Two objections stand independent of the fairness-criterion dispute, and they are arguably more serious: **The label is contaminated.** These tools predict *rearrest*, not offending. Rearrest is a function of policing intensity as much as behavior — the same measurement problem as [[Predictive Policing]]. A tool perfectly predicting rearrest is partly predicting where police concentrate. **The inputs encode structural disadvantage.** Employment stability, residential stability, education, and family criminal history are all strongly correlated with race and class in the United States. An instrument excluding race but including these is performing [[Statistical Discrimination]] through proxies — and, uncomfortably, it is scoring people up for being poor. ## The deeper objection Even a perfectly calibrated, unbiased instrument raises a question the mathematics cannot touch: **is it just to detain an individual on the basis of a group-level statistical propensity?** Anglo-American criminal law nominally punishes acts, not predicted acts. Actuarial sentencing punishes membership in a reference class. A defendant told they are being detained because 30% of people resembling them reoffend has no way to contest it — they cannot demonstrate they are among the 70%, and the instrument makes no claim about them individually. Wisconsin's *State v. Loomis* (2016) upheld the use of COMPAS at sentencing while requiring cautionary instructions and barring it as the determinative factor — a compromise that permits the practice while conceding the concern, and that depends on judges weighting a score appropriately, which is precisely the discretion the tool was adopted to constrain. See [[Metric Capture]] on why scores presented as advisory tend to become dispositive in practice. ## Related [[Fairness Impossibility Results]] · [[Predictive Policing]] · [[Statistical Discrimination]] · [[Credit Scoring]] · [[Algorithmic Bias]] · [[Metric Capture]] · [[Mathematics and Power Hub]]