Assessing Program Outcomes can be Tricky

Reclaiming Futures (2011). Blog: News & Information. Portland, OR: Reclaiming Futures, Portland State University.

Many people believe that agencies can assess their effectiveness entirely with pre/post comparisons of youth outcomes, such as recidivism or drug use before and after treatment. Apparently, they do not know about the statistical bias present in that sort of comparison. These sort of pre/post measures can be distorted by the statistical bias known as a “selection-regression artifact.” The concept was explained very well in 1980 by Michael Maltz and his colleagues, but since I don’t know many people who enjoy reading articles about research methods written more than 30 years ago, I’ve provided a summary in this video:

However, if you are the type of odd person who enjoys reading about evaluation methods, here is the full citation:
Maltz, Michael D., Andrew C. Gordon, David McDowall, and Richard McCleary (1980). An Artifact in Pretest-Posttest Designs: How It Can Mistakenly Make Delinquency Programs Look Effective. Evaluation Review 4(2): 225-240.