Sentencing and the Wisdom of Crowds

One of the major justifications given for developing sentencing guidelines has been the diversity and resulting disparity of sentences produced by judges for similar offenders committing similar offenses. A common practice in a state considering guidelines has been to bring a bunch of judges together (sometimes with prosecutors and defense counsel), give them all some real or hypothetical cases, and then ask them to pronounce sentence. Inevitably you would get some bizarre outliers, the same sort you would see examining real sentencing data. Based on these outliers, a case could be, and often was, made that greater uniformity was necessary and thus sentencing in that state should be more structured.

It's difficult to argue. Due process, equal protection, just basic decency--all require that offenders of a similar type receive similar sentences for similar offenses. But, in making everyone wear a diaper because a couple of people pooped, I wonder it we didn't miss the real significance of those exercises. How many of the scenario sentences were the same or very similar? That was rarely reported or even considered. There are outliers in almost every thing, hence the term. (Hence, Salma Hayek.) But maybe the real news was the usual sentence that most of the participants got close to.

A few years ago James Surowiecki published a book called The Wisdom of Crowds. While not universally convincing, he made the case that collective judgments frequently (perhaps usually) surpass the estimates of experts and professionals on topics. The usual example is the "guess the whatever" at a fair--the number of jelly beans in a jar, the amount of pennies in another jar, the weight of the fat guy. While any one guess was likely wrong (and some would be notable outliers), the average of the guesses, given enough people, was usually very close or even exactly right.

The sentencing scenarios were really "guess the sentence" games, telling us as definitively as practiced judgment could what a proper sentence was. In many cases, those sentences or ranges off them formed the basis for the eventual guidelines in the state. (In practice, such as in MD, what often happened was that a small group of judicial leaders became despondent, took the results into a room, and came back out with what they said the proper guidelines should be.) But the results tended to be sold based on the idiocy of the outliers, not on the wisdom of the judicial crowd. So, we lost an opportunity to build a foundation for the study of sentencing on reiterations of reiterations of these kinds of sentencing experiments around the nation to develop national professional parameters for sentencing.

If you think about it, our current sentencing data, by state or collectively as a nation, is a giant "guess the whatever" exercise for correct sentences of similar offenders committing similar offenses. Especially in states with commissions created in part to develop sentencing data bases, we've developed a massive amount of collective experience, "wisdom of crowds." (The irony is that, the more structured and prescriptive, the less valuable unless the guidelines were those based on the exercises instead of "top-down" policymakers or those self-assured judges coming out of the room.) What I would like to propose here is that we take this collective wisdom out for a spin.

We have the data now, and funding sources seem to be coming online, to actually pull together our sentencing guesses and to build on their "wisdom." All states, commission or not, if they have the data available, should now agree to partner in data sharing and pooling to generate "crowd" sentences for their similar offenders (however minutely defined within this gigantic data set) and then to analyze the offenders' subsequent recidivism, violence, or other public safety concerns. We could also do a much better job of replicating on a national level the cost-effectiveness work of the Washington State Institute for Public Policy. In fact, the foundation dollars now moving forward could either create a national sentencing WSIPP protected in a university or expand Vera for the function to do exactly these kinds of national studies. (I would go further and create a national board of professionals similar to a Cincinnatus Committee discussed in my California Challenge in earlier posts to vouch for and buffer the results to give them legitimacy, credibility, and protection, but others may feel that an unnecessary step.)

In any case, it is clear that our available pools of sentencing data contain enormous untapped potential, years and decades of "guesses" that, if Surwiecki is right, could guide effective policymaking at a time when some violent crime seems to be increasing again (assuming anyone really wants to make effective policy). As I mentioned, some of the most effective commissions have founded their guidelines precisely on this notion, killing off outliers (statistically) and taking interquartile ranges or one standard deviation each way from the mean as their recommended sentencing ranges. Plus, the arguments for linking sentencing to demonstrated data would be even stronger, even for states scared of or unimpressed by commissions and guidelines.

What do you think? Let's "open source" this, a definite "wisdom" approach. How could we frame this to get it going, get the necessary funding and support? The floor is open.