The second of a two-part series. We're inviting participants in sentencing commissions to contribute their perspectives in an ongoing dialogue with our readers. In this series David Wright, former Director of Research for the Oklahoma Criminal Justice Resource Center, which staffs the state's commission, talks about being a research director, particularly the hows and whys of sentencing databases. We hope that other participants will use this as an opportunity to share your views and knowledge in the future. The first part started a listing of the requirements for doing good commission research, focusing on good data. This conclusion continues that discussion and examines the other requirements.
The Research Director and staff should know and acknowledge database weaknesses and limitations. This should be done in all work as part of the up-front methodology. This should involve discussions about why the limitation(s) exist, which may not be a fault of the research staff and may really be beyond the research staff’s capabilities or duties, such as weaknesses in the criminal justice system. Documentation should also show what is being done to improve or address the limitations. In this methodology section, it is also important to list the database/methodological strengths and to build on them. I highly recommend developing a list of strengths and weaknesses revolving around assumptions made about the database so that everything is documented. This process should make it easier to avoid over-stating your findings and conclusions. It should make it less likely for the data to be over-generalized. Because of the data and research limitations, avoid leaps of logic from statistical data or findings, to statements or policy recommendations that the finding is specifically and directly caused by something, or that even make it appear as though something were a direct cause (unless, of course, there is a direct cause and effect).
It is not necessary for the data in the Commission database to include every case for the information produced to be valid and useful, especially in the case of research reports or analyses of trends. Focus should revolve around patterns and percentages and not individual counts. Nevertheless, there will likely be criticisms that the data are incomplete or inaccurate without an accounting of all cases. Ideally, it would be good to collect all cases, especially for prison population projections.
In attempting to develop a comprehensive analysis of sentencing practices, it is necessary to move beyond strictly sentencing data and consider the broader criminal justice system. Crime and arrest data are of particular relevance to the sentencing practices of the state, as they influence or determine the possible felony conviction pool. Crime and arrest data should be easily obtained from the state’s bureau of investigation.
With regard to the Commission’s database, it is important to guard against Commissioner’s selective use and interpretation of data. These types of Commissioners want to have it, the data and analysis, both ways. These Commissioners will bash your information as bad data or research when it does not back or support their positions and will endorse the database and information when the information ends up supporting their opinions.
The data are the foundation. It really does not matter how good the staff, the research director, or the reports (quality and layout) are. If your data are not good, it will bring down or hold back the whole Commission.
Relevant Research
Following striving for high quality data, a critical element to being a successful Research Director for a Commission is making the research relevant. This can be accomplished through several methods. First, select timely topics to research. A few influential Commission members could help you develop an agenda, which could then be approved by the Commission. This will provide you with cover as you work on the topics, as well as giving you direction from someone other than yourself, the research staff, or the Executive Director. This will allow you to avoid “losing” research topics (death penalty) and hopefully will provide you with some topics you can “win” or at least will give you cover.
Secondly, research and analyze topics that are critically important, such as legislative bill impacts and prison population projections. This will ensure an important policymaking audience, which will be reminded that the Commission is vital. With this audience in mind, keep the reports, presentations, or messages, short, simple and straightforward.
Third, consider your message, style, and delivery and remain neutral. Often times the research speaks for itself and allows you to avoid the risk of seeming or becoming an advocate for a particular position. Remaining neutral will also assist you in not being confrontational and building a cumulative opposition. At some point in the sentencing research there is a point of diminishing returns. The longer you have your job, the greater chance you have that you will offend someone. Eventually, you may offend everyone. However, the research results should be presented in a manner that revolve around the question or hypothesis and why it is important and not a particular slant.
Finally, all analysis, research, and/or data should be filtered and/or reviewed prior to release. It is advised that the Commission and/or host agency have a data release policy and/or information request policy. Then make sure that requests are in writing. Have all information used to meet that request go out only after it is packaged. Anything released can come back to haunt you. You should consider anything that leaves the office, particularly something that was originally requested, open for criticism and critique. Often times you may not even know for sure how the information you released is going to be used or who else will also end up receiving it. Consequently, it is important that the information is accurate, objective, and packaged in a non-offensive manner. Follow-up is advised in order to see how the information shared was utilized and to see if it was re-packaged. Once a positive relationship and a high comfort level is established with the requestor of information, then the follow-up and feedback should become more routine.
Stupendous/Superior Staff
Another important element in producing quality data and reports is the staff, which should be very resourceful. A staff person who can serve as the database manager and provides database maintenance and data cleaning is critical. This person should have the ability to perform matching and merging of databases including the Commission database and others, in order to combine multiple years of information and to link with other databases. This will allow the opportunity to perform longitudinal research, as well as enhance the ability to compare and analyze additional data sources.
The quality of the data and the research will also benefit from having a statistician or someone with a great deal of background in statistics. Ideally, this person will also have the analytical abilities necessary to truly analyze and make sense of the data. This will involve turning the data produced from a statistical package, such as SPSS, SAS, SQL, Access, or Excel, into straightforward, simple, but useful reports and information examining sentencing practices. The goal should be to develop professional data-driven research that is relevant.
Tons of Time
One factor that you will need as Director of Research for a Commission is time, ideally, three years. This will give you and the research staff a chance to develop experience and expertise. Time will provide your staff with the opportunity to learn from mistakes and implement adjustments. Three years will give the Executive Directors and the research reports some time to have a chance for the messages to sink in and for, at least, incremental changes to occur. Ultimately, time will give you and the Commission a chance to build credibility.
Thick-skinned
Conservatives, liberals, democrats, republicans, the lock ’em up and throw away the key types and the give everyone a second chance types, will all want you to run numbers that will support their position and agenda. The temptation, especially as a new Commission or Research Director, is to try and please everyone by giving them what they want. This seems particularly to be the case because anytime you are producing numbers, making presentations, and performing program evaluations that people do not agree with. They will criticize the data, the methodology, the research, and ultimately you. People want “your” numbers to support their side. When they do, then they love you. In other instances, when the numbers do not support their side, then you are wrong; you are at fault in some manner. These types of people will speak out of both sides of their mouths and want it both ways.
What you have to remember is to test hypotheses, trends, etc. Force requestors to put their requests in writing. It will make them state what they are asking and what they want to know; what they are trying to find out. Once the information they are hoping to find has at least been turned into a real research question - a hypothesis - then it is a matter of assessing its testability and its researchability.
Conclusion
I have often heard it said, from prominent legislators, that criminal justice and sentencing in particular is the one arena where legislators believe that they are the experts and what the practitioners (judges, district attorneys, defense attorneys, etc.) and real experts (i.e., researchers, corrections officials, scientists, etc.) say does not matter - that it will not influence policymaking. While there may be more truth to this statement than I would care to believe, I think that the suggestions described in this chapter will, at a minimum at least, allow and/or increase the possibility for there to be a link between research and policy.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment