by Jeffrey Butts
December 5, 2018

This morning, the Trump Administration abolished the Science Advisory Board (SAB) for the Office of Justice Programs at the U.S. Department of Justice. The news was not entirely unexpected for the members of the Board (including me). Since the Obama Administration left town, there had been just one meeting of the SAB. The next three meetings were never scheduled. With no official word about the Board’s fate, most of the members expected to hear that the SAB’s official charter would be allowed to lapse. This morning, the word came. There will be no more Science Advisory Board for the Office of Justice Programs.

The SAB was formed by the Obama Administration in 2010. Its goal, according to the 2015 edition of the SAB charter, was to provide advice on science-related issues.

“The Board will provide the office of the Assistant Attorney General (AAG) of the Office of Justice Programs (OJP) with valuable advice in the areas of social science and statistics for the purpose of enhancing the overall impact and performance of its programs and activities in the areas of criminal and juvenile justice. The Board will help develop long-range plans, advise on program development, and recommend guidance to assist in OJP’s adherence to the highest levels of scientific rigor as appropriate. The Board will provide an important base of contact with the criminal justice and juvenile justice academic and practitioner communities.”

The membership of the Board, including founding Chair Al Blumstein and his successor Ed Mulvey represented a wide range of academic disciplines, including social and behavioral sciences, as well as professional disciplines including law enforcement, corrections, treatment specialists, prosecutors, and the defense bar. Some of the initial members included William Bratton, Frank Cullen, Mark Lipsey, Tracey Meares, Joan Petersilia, Rick Rosenfeld, Rob Sampson, and David Weisburd. The most recent membership list maintained that tradition.

During their meetings from 2011 to 2017, the members of the SAB received updates from the leadership of OJP and its bureaus (e.g., NIJ, BJS, OJJDP, BJA) on the Office’s initiatives and funding plans. The Board members, in turn, would provide or facilitate presentations about pressing issues in policy and practice and the extent to which available solutions were grounded in science or could be better supported by new evidence. The members of the SAB, who served with no compensation, did not judge or recommend particular OJP policies or programs. The mission was to provide guidance as available from the scientific literature and the newest findings of evaluation research.

At the first meeting, then Assistant Attorney General Laurie Robinson told SAB members she hoped the Board could play six key roles in support of OJP.

• First, to look at the broad role of science within OJP and how we can better integrate what we learn from science in to our programmatic design and spending.
• Second, to think about ways, of course, to strengthen the research and statistical functions within OJP.
• Third, to suggest broad priorities on which research might be focused.
• Fourth – and I think this is a critical area, particularly in light of certain events at OJP in the last decade – to consider and make recommendations about institutional ways to protect the science here going into the future. We may not enjoy the support for science in future Assistant Attorneys General and Attorneys General that we have today. DOJ’s is a lawyer culture and we know from history that it can be hostile to science. We need to build in protections.
• Fifth, to think about ways that OJP can more effectively connect researchers with practitioners and policymakers – particularly on translation of evidence – and generally serve as an avenue for outreach to the field and a promoter of a two-way dialogue.
• And finally, to play a helpful role in providing advice and counsel to us on practical concerns, like improving the way we handle peer review.

The members of the SAB were very conscious of Robinson’s fourth point, and during 2016 the Board devoted considerable effort to preparing advisory notes for the field. We joked (before we knew the outcome of the 2016 election) that these advisory notes represented our “message in a bottle” that would be set afloat on the broad ocean of political change. We hoped they would be of service to future researchers and policymakers and provide a primer of sorts on the role of science in justice policy and practice. The statements represented the consensus views of the diverse membership of the Board, were approved by a vote of the Board, and then promulgated on the Justice Department’s website on January 17, 2017.

Advisory Statement 1 clarified the important but limited role of randomized controlled trials in the development of effective policies and practices, including:

Randomized controlled trials (RCT) generally provide the strongest or most defensible causal evidence for programs and practices, but it may not always be possible to implement successful RCT evaluations in the field. Many important questions in the field of justice are not answerable using RCT studies—either for practical, economic, political, or ethical reasons. Research questions that are very difficult or expensive to answer using experimental methods may merit the necessary investment if they have widespread or profound social consequences, just as research questions with only modest consequences still merit experimental investment if they can be answered easily and at little cost. Funding for RCT evaluations should be managed like an investment portfolio with resources concentrated on the most effective combinations of theoretical salience, research feasibility, and social benefit.

Advisory Statement 2 focused on the role of evidence in justice policy and practice, including:

The strength of evidence required to judge the value of programs and practices in the justice field is a question of balance. Judgments should be based on the best available evidence, but the strength of evidence required for any decision is gauged by the costs of error and the burden of increasing evidentiary quality. Decisions with little consequence require less accurate evidence and less exhaustive evidence. Highly consequential decisions require more evidence. Navigating the continuum of evidence-supported decision-making is complex and subjective. The available evidence for any policy, program, or practice is not the product of a straightforward and untrammeled search for effectiveness. It emerges from a contentious and inherently political process that governs social investment in research.

Advisory Statement 3 examined the responsibility of policymakers and researchers to both identify and develop effective interventions and strategies, including:

Justice programs are generally designed to achieve specific purposes, and produce intended effects. The effects of an intervention on the target population, both intended and unintended, are known as outcomes. Recidivism is the most common outcome of interest within the justice field, but other measures are used, including satisfaction with services, impacts on drug use, employment, educational success, and the legitimacy of criminal justice agencies across communities. Identifying the correct outcomes for a policy, program or practice is a basic task associated with program development, performance measurement, and program evaluation. … There are a number of considerations related to outcome measurement that OJP must consider in drawing inferences about the effectiveness of justice programs. These include: primary and secondary outcomes, unintended outcomes, cost/benefit, implementation fidelity, and efficacy versus effectiveness research.

The SAB and its meetings and products supported scientific thinking inside the federal justice structure with the hope that all aspects of criminal justice, juvenile justice, and the much larger concerns of social justice would be informed by evidence. It was an effort to make the production and use of future evidence sensitive to established, scientific considerations that did not simply mirror the preferences of current office holders.

I hope that kind of thinking returns one day soon.