Before you demand a “data-driven” justice system, make sure you know who is actually behind the wheel.

Jeffrey Butts, August 2020 (rev)

Someone pays for the evaluation evidence used to fashion crime control strategies. Evaluation research tends to be expensive. Some statistical work may be conducted by analysts operating on their own, but evaluation research is labor intensive, often requiring new data collection and outside funding.

When we review the findings of evaluation research, we are seeing answers to whatever questions researchers were paid to investigate. What sort of questions were not asked? How does this affect justice policy and our understanding of the best ways to ensure community safety and community well-being?

Evaluation evidence comes from investments made by policymakers, government agencies, and foundations. Their investments are not free of bias. They reflect the goals, beliefs, and values of funding bodies as shaped by cultural, class-based, and racial biases that tend to favor non-structural explanations of crime. Foundation officers and elected officials locate the origins of crime in individual pathology rather than inequality, injustice, and community disinvestment. 

As a result, most justice research tests the effects of interventions on individual behavior despite the reality that behaviors are shaped by social and community context.

Are wealthy neighborhoods relatively free of violence because non-violent people decide to live there, or do structural and economic advantages lead to lower rates of violence? Reasonable answers to this question would suggest a range of public safety policies, but research tends to concentrate on one approach– individual interventions. 

Evaluation researchers are also rational creatures. Their main goal is to publish–a lot. They prefer to evaluate interventions that can be tested quickly with administrative data so they don’t have to spend time collecting new data. Community-level interventions and prevention programs are time consuming, hard to control, and more likely to produce ambiguous findings. It’s a slow, labor intensive road that may lead to fewer publications and less exciting results. So, like their funding partners, most evaluation researchers prefer to address crime and justice issues at the individual level, testing therapeutic interventions and law enforcement strategies.

When someone tells you “what research says” about how to reduce crime and violence, remember they’re describing a research base created by people and organizations with opinions, values, and self-interest.

Research findings don’t appear like wildflowers in a meadow. They are planted and watered by gardeners with intention.