Before you demand a “data-driven” justice system, make sure you know who is actually behind the wheel.

Research evidence is not simply discovered; it is purchased by the government agencies and philanthropies that pay for evaluation research.

Evidence does not emerge from a pristine and impartial search for truth. It is subject to the political economy of research. Evaluations of justice interventions can be very expensive. Some statistical analyses may be conducted by academics operating on their own, but good evaluation research often requires new data collection and this can be expensive, usually requiring outside funding.

This means our evaluation evidence comes largely from investments made by policymakers, government agencies, and foundations. These investments are not free of bias. They reflect the goals, beliefs, and values of funding bodies as shaped by cultural, class-based, and racial biases. When we review the findings of evaluation research, we only see answers to the questions researchers were paid to ask. What sort of questions do you think funding organizations avoid? How does the lack of such information affect justice policy and community well-being?

One of the most influential sources of bias relates to the level of measurement used in evaluation research. Most justice research tests interventions on individual behavior, although behavior is obviously shaped by social and community context. Are wealthy neighborhoods relatively free of violence because non-violent people decide to live there, or do structural and economic advantages lead to lower rates of violence?

Reasonable answers to this question would suggest a range of public safety policies, but research funding tends to focus on one approach– individual interventions. In part, this may be due to the greater complexity of community-level evaluation designs. It is also clear, however, that foundations and elected officials tend to favor explanations of crime that point to individual pathology rather than inequality and injustice. It fits their worldview more comfortably.

Moreover, evaluation researchers are rational creatures. Their main goal is to publish–a lot. This means they prefer to evaluate interventions that can be tested with administrative data so they don’t have to spend their own time collecting and creating data. And, they know it is easier to generate strong effects from the larger sample sizes made possible with individual-level interventions. So, like their funding partners, they prefer to address crime and justice issues at the individual level, mainly with therapeutic interventions and law enforcement approaches.

Community-level interventions and prevention programs take too long, are too hard to control in evaluation designs, and are more likely to produce ambiguous findings. So, why take the risks involved in evaluating prevention and community change. It’s a slow, labor intensive road that leads to fewer publications and less exciting results.

All these factors shaped today’s evidence base for crime, justice, and public safety. So, when someone tells you “what the research says” about how to reduce crime and violence, try to remember they’re describing the research base as it was created by people and organizations with opinions, values, and self-interest. Research findings don’t just appear like wildflowers in a meadow. They are planted and watered by gardeners with intention.