The reasons behind using data to inform policy can vary from office to office. We ask three agencies that have partnered with Prosecutorial Performance Indicators to share their insight into why they decided to get a better look at the numbers.
San Joaquin District Attorney’s Office (California)
Why is it important for your office to use data when creating policies and practices?
Understanding operational data and the impact public sector activities have on our communities is critical to good governance. Too often policies can be driven by the loudest voices, not the most urgent needs. Taking the time to look at data about our operations and the impact our work is having is critical because the data highlights the areas that need the most work. For example, in San Joaquin County, domestic violence cases account for 20% of all misdemeanor filings. With misdemeanors, even if successfully prosecuted, those individuals will be back in their communities within a very short period of time – and often back in the homes where the violence first occurred. We are using the data as a catalyst to have broader conversations with prosecutors and victim rights advocates on new options for prosecution that continue to hold individuals accountable for the harm that has been caused but also puts in place a range of interventions to heal and address the root causes of the violence in the first place.
What has been the most surprising data trend that you’ve found through working with PPI and why?
To put things in context, everything about 2020 was a surprise, so it is difficult to know where to start. In general, working with the PPI team has given us a new language to talk about the data we have and to start discussing what we also need to be paying attention to. More specifically, we have a better lens for looking at core measures of community safety. In 2020, we found the number of referrals for cases dropped overall and correspondingly, the number of cases filed. But more interestingly, the percent of cases filed for felony charges rose significantly accounting for 36% of filings, compared to an average of 28% of filings in prior years. Whether this is a unique fluke in a complex year is still unknown – but what we have been saying in our office, and training prosecutors on, is the importance of focusing prosecution efforts on the drivers of serious crime and violence. Hopefully, this data is an early indication that we are doing a better job of focusing our resources where they are needed most.
What is an example of a policy or practice your office has created that was guided by data?
Data over the last year has shown an upward trend in the seriousness of domestic violence. So while the overall number of DV case referrals remained relatively flat in 2020 (or perhaps underreported); of the cases that were referred the level and intensity of violence was higher than in prior years. We saw far more cases of assaults with serious injuries, use of weapons, guns, strangulations, and sadly, homicides than in prior years. Using this data, we are forming a Domestic Violence Response Team in partnership with local law enforcement agencies to provide wrap-around services to families. The target population are families/couples for whom law enforcement responds to regarding a domestic disturbance, but on arrival have no reason for an arrest. Times are complicated and we understand that many families are struggling. The program will offer services and supports for the whole family – addressing life stressors around employment, housing insecurity, and access to healthy food. The goal is to stabilize families in crisis and avert a potential escalation in violence. The project was approved by our local Board of Supervisors based on data we were able to share about the urgency of the situation and the escalation of harm that was occurring.
What are the biggest impediments to prosecutors using data to help decision making and how can they overcome them?
Really the biggest impediment to using data is a lack of uniform, accessible data to reflect and compare against. Over the past several decades, most criminal justice data has focused almost exclusively on crime reports: how much, where, what type, who is committing crimes and getting locked up for crimes. There has been very little discussion on prosecution: What constitutes good prosecution, who is involved in making prosecutorial decisions and setting prosecution policies, and most notably what is the impact of prosecuting and sentencing crimes. In other words, are we meeting our core objectives of making communities safer?
Fortunately, groups like PPI and others are starting to not only raise these questions, but provide tools and resources so that ordinary, underfunded county prosecutors in small and medium-sized communities across the nation have the opportunity to look at their own data and benchmark it against other places. The list of 55 indicators gives us a way to think critically about what we really want to be achieving as prosecutors and how we will track our actions and see if we meet our goals and objectives.
If you had one piece of advice for other prosecutors considering integrating more data in their decision-making process, what would it be?
Just start! There will never be a good time. You will never have good enough or clean enough data. Just start with what you have – even if it is only a handful of indicators that you will be measuring. Start by measuring what you can and then sharing that information with internal and external stakeholders. You will know by the questions they ask what you need to work on next.
The San Joaquin County District Attorney’s Office is dedicated to advocating for victims, ensuring public safety and equitable justice for all. Our 2020 Data Report can be found at SJGov.org/DA.
Philadelphia District Attorney’s Office (Pennsylvania)
Why is it important for your office to use data when creating policies and practices?
Using data to inform policies and practices make it easier to track the progress of reform initiatives and, eventually, their impact. Especially when you are making many changes concurrently, data help understand how different efforts will impact the office operationally in terms of case flow and caseload, and ideally identify related processes and practices that should change to reinforce or build on existing policies. When data are used to inform policy, that increases the likelihood that metrics have been identified to assess how the policy is going, making it easier to iterate, adjust, and evaluate longer-term.
What has been the most surprising data trend that you’ve found through working with PPI and why?
“It’s important to not be surprised by the values of the metrics on their own. PPIs–and any metrics like them–are useful for identifying issues that may be worth investigating but are insufficient for drawing conclusions. Instead, they should serve as a starting point for discussions with attorneys and more focused research. My job on the PPI project is to create and evaluate metrics. As an analyst in the DATA Lab, my job is also to engage in focused research, some of which is motivated by what metrics like these show us.”
– Sean Mason, Lead Data Analyst in the DATA Lab working on PPIs
What is an example of a policy or practice your office has created that was guided by data?
Data were used to ensure that both Bail Reform (February 2018) and Supervision Reform (March 2019) would impact many people, and have a big impact on mass supervision, respectively. For bail reform, we analyzed 5 years of data (2013-2017) for more than 200,000 lead charges to identify 25 charges that accounted for roughly two thirds of all lead charges during that time period. An external evaluation of the policy found that – due to policy exceptions not accounted for in initial analysis – 60% of cases were actually eligible for the reform (Ouss and Stevenson, 2020: 10). Still, a policy for which 3 in 5 cases are eligible has the potential for a large impact.
With supervision reform, counterfactuals were created to understand how different permutations of the policy would impact future years of supervision. This allowed us to adjust the policy with specific goals in mind, e.g., cutting future years of supervision in half over a certain period of time. Our evaluation of this policy found that “42% fewer years of community supervision were imposed in the first two years of the Krasner administration than in the two years prior” (Tran et al., 2021) (bit.ly/EndingMassSupervision). An external evaluation of this policy is underway.
What are the biggest impediments to prosecutors using data to help decision making and how can they overcome them?
Broadly, lawyers don’t necessarily receive training on how to use and interpret data, which is a big reason we are excited about partnering with the PPIs. Just as I feel less comfortable talking about the law because I only took 1 law school class as part of my criminology training, the same may apply to lawyers – are there classes in law school on how to use data (and research) to inform legal advocacy, policy, or how to distinguish between robust and flawed data (and research), and more? Relatedly, on the data side, we need to create both shorter-term and longer-term feedback loops for attorneys, balancing quicker feedback on some decisions while acknowledging that impact of contacts with criminal legal systems can take a while to understand even with our most basic metrics (e.g., recidivism). My hope is that prosecutors see data as helpful for situating day-to-day decision-making in the context of longer-term trends.
If you had one piece of advice for other prosecutors considering integrating more data in their decision-making process, what would it be?
Partner with the PPIs or other academics who are able to help complement internal resources and provide structure, comparative perspective, and accountability for your work. If you have internal data capacity, build data work into the process of developing new policies and practices, and then evaluating how they are going. This can entail only implementing policies that have supporting data or research, iteratively reviewing data on how policies are going following implementation and using data to justify changes based on those results.
Multnomah County District Attorney’s Office (Oregon)
*This office recently partnered with PPI and is early in the data collection and analysis process.
Why is it important for your office to use data when creating policies and practices?
Departing from the harmful and outdated model of equating justice with convictions, DA Schmidt is actively working towards performing to public safety outcomes based on data over performing to ‘who can get the longest sentence’. PPI is among the mechanisms that allows us to do this. The importance of letting data drive our policies and practices is rooted in both values and efficiency. Public safety and transparency are among our top priorities. If we’re spending time and resources that fail to achieve this end we’re not only not helping, we’re causing potential harm. Data provides an opportunity to hold a mirror up to our work and better understand if our mission is aligned with our outcomes.
What are the biggest impediments to prosecutors using data to help decision making and how can they overcome them?
Our office has a forward thinking culture, seeking to modernize our practices and increase data transparency to maximize public safety outcomes. Not every office has this culture. The criminal justice system is historically opaque to the public. As the public seeks greater accountability from the system, we must rise to the occasion and provide metric outcomes for the work we do in addition to meaningful outreach to the community to ensure our priorities align.
If you had one piece of advice for other prosecutors considering integrating more data in their decision-making process, what would it be?
Solicit feedback from the community and partner with community-based organizations that represent justice impacted people. As enlightening as data may be, it can also be alienating without the story behind it.
Josh Vaughn is an award-winning freelance reporter who has covered criminal justice issues in Pennsylvania for the last decade, most recently with the criminal justice journalism website The Appeal. Readers may follow him on Twitter @Sentinel_Vaughn.”
Recent Comments