Is there Hope for Evidence-Based Policy?

Vital City magazine and the Niskanen Center’s Hypertext have a special issue on the prospects for “evidence-based policymaking.” The issue takes as its starting point, Megan Stevenson’s Cause, effect, and the structure of the social world, a survey of RCTs in criminology which concludes that the vast majority of interventions “have little to no lasting effect.” The issue features responses from John Arnold, Jonathan Rauch, Anna Harvey, Aaron Chalfin, Jennifer Doleac, myself, and others. It’s an excellent issue.

My contribution focuses on the difference between changing preferences versus constraints. Here’s one bit:

Some other programs that Stevenson mentions elsewhere are also not predominantly constraint- or incentive-changing. Take, for example, the many papers estimating the effect of imprisonment on the post-release behavior of criminal defendants via the random selection of less and more lenient judges. At first, it may seem absurd to say that imprisonment is not about incentives. Isn’t deterrence the ne plus ultra of incentives? Yes, but the economic theory of deterrence, so-called general deterrence, is rooted in the anticipation of consequences — the odds before the crime. By the sentencing stage, we’re merely observing where the roulette wheel stopped. Criminals factor in the likelihood of capture as just another cost of doing business. Thus, the economic theory of deterrence predicts high rates of recidivism, as the calculus that justified the initial crime remains unchanged after punishment. To be sure, imprisonment might change behavior for all kinds of reasons. Maybe inmates learn that they underestimated the unpleasantness of prison, but perhaps they improve their criminal skills while in prison or join a gang, or perhaps the stain of a criminal record reduces the prospect of legitimate employment. Thus, the study of imprisonment’s effects on criminal defendants is intriguing, but it’s not testing deterrence or incapacitation, on which we have built a body of work with clear predictions.

Indeed, on Stevenson’s list only hot-spot policing is a clear example of changing constraints. It is perhaps not coincidental that hot-spot policing is one of the few interventions that Stevenson acknowledges “leads to a small but statistically significant decrease in reported crime in the areas with increased policing.” While I do not begrudge Stevenson her interpretation, other people shade the total evidence differently. Here, for example, is the Center for Evidence-Based Crime Policy, in my experience a rather tough-minded and empirically rigorous organization not easily swayed by compelling narratives:

As the National Research Council review of police effectiveness noted, “studies that focused police resources on crime hot spots provided the strongest collective evidence of police effectiveness that is now available.” A Campbell systematic review by Braga et al. comes to a similar conclusion; although not every hot spots study has shown statistically significant findings, the vast majority of such studies have (20 of 25 tests from 19 experimental or quasi-experimental evaluations reported noteworthy crime or disorder reductions), suggesting that when police focus in on crime hot spots, they can have a significant beneficial impact on crime in these areas. As Braga concluded, “extant evaluation research seems to provide fairly robust evidence that hot spots policing is an effective crime prevention strategy.”

Indeed, I argue that most of the programs that Stevenson shows failed, tried to change preferences while those that succeeded tend to focus on changing constraints. There are lessons for future policy and funding. Read the whole thing.