As a result of Washington State’s evaluation of evidence-based programs that showed the substantial relationship between implementation quality and youth outcomes,213 the state legislature directed the
Washington State Institute for Public Policy (WSIPP) to develop required program quality and outcome standards for all state-funded juvenile justice programs. As stated in the legislation, “Because model adherence and competent delivery of research-based intervention programs is critical for reducing recidivism, the Washington State Institute for Public Policy shall develop adherence and outcome standards for measuring effectiveness of treatment programs referred to in this act. The [Juvenile Rehabilitation Agency] shall utilize these standards to assess program effectiveness. The courts shall also utilize these standards in determining their continued use of these alternatives. The courts shall not continue to use programs that do not comply with these standards.”
The standards developed by WSIPP to guide the implementation of juvenile justice services include requirements in the following areas:214
• Program and organizational management practices
• Staff hiring criteria, and staff training, performance evaluation, and retention practices • Development, use, and maintenance of a manual that specifies the program policies and
procedures for staff as well as organizational quality-assurance activities
• Identified outcome measures and expectations including at least 75 percent successful program completion rates, improvements in dynamic risk and protective factors, and reduced recidivism rates with sufficient follow-up to assess both further juvenile justice system and adult corrections involvement.
Washington employs quality assurance specialists to help promote these standards throughout the state and a policy committee to provide ongoing oversight. As further accountability measures, the oversight committee guides comprehensive empirical evaluations of program providers, directs the specialists to employ corrective actions as needed, and can discontinue funding for any program that fails to comply with these quality standards.
2. Assess service quality and fidelity to the research. Juvenile justice systems that commit to adopting
programs and practices demonstrated by research to improve youth outcomes and that establish quality standards for guiding the provision of these services need an objective way to determine whether community- based providers and facilities are meeting these standards. States and counties are increasingly using
standardized tools to conduct system-wide assessments of service quality, such as:
■ Standardized Program Evaluation Protocol. The Standardized Program Evaluation Protocol (SPEP)
is a scoring tool for assessing program quality based on the findings from over 600 studies on what works to reduce juvenile recidivism. The SPEP helps measure service matching, whether the program model reflects an effective service approach, service dosage, and implementation quality. Higher program SPEP scores are correlated with 10- to 15-percent lower recidivism rates. Counties in Arizona, Connecticut, Delaware, Florida, Iowa, North Carolina, Pennsylvania, and Wisconsin are using the SPEP to guide program selection, implementation, and investments.215
■ Correctional Program Checklist. The Correctional Program Checklist (CPC) is another tool that
juvenile justice systems can use to ensure program investments improve youth outcomes. In 2003, for example, the Oregon legislature required state agencies that provide treatment to reduce juvenile delinquency to annually increase the percentage of programs that are evidence-based. The Oregon Youth Authority used the CPC to define and identify those programs that met the criteria for “evidence-based.” As of June 2012, approximately 80 percent of correctional facility living units and 98 percent of contracted community-based residential programs in Oregon met the CPC criteria of “Effective” or “Highly Effective.”216
■ Correctional Program Assessment Instrument. The Correctional Program Assessment Instrument
(CPAI) is used to measure how closely a correctional program adheres to the research on what works to improve outcomes for youth. The RECLAIM Ohio Initiative was used to pilot the CPAI through the University of Cincinnati, and Ohio continues to use the tool to guide and evaluate the implementation of juvenile justice programs across the state.217
Juvenile justice systems will need to make a number of key decisions regarding the use of these service quality assessment tools, including:
■ Whether to develop internal systems capacity or external partnerships to conduct the assessments,
and how often to conduct such reviews
■ How to encourage provider buy-in so assessments become a quality improvement tool rather than
engendering rote compliance
■ How to electronically capture the assessment results
■ How to use the results to improve services, hold providers accountable, and allocate funding
Juvenile justice systems that address these operational challenges in systemic ways, including devoting sufficient staff and resources to the assessment process, and that do so in partnership with the provider community, are likely to maximize the benefits of their service investments.
3. Provide ongoing, expert implementation assistance. In order to achieve expected service quality
standards, providers require ongoing support to address quality-improvement needs.218 A number of states
have experimented with a range of systemic solutions for connecting service providers with ongoing and comprehensive assistance with program implementation:
■ In Connecticut, the Department of Children and Families and the Court Support Services Division of
the Judicial Branch jointly fund the Connecticut Center for Effective Practices to provide training and quality assurance for and to help evaluate all MST programs in the state.219
■ Florida contracted with a private intermediary to assist with the local procurement of services, service-
delivery oversight, and quality assurance for the provision of evidence-based practices statewide as an alternative to confinement.220
■ Ohio established the Center for Innovative Practices to provide training, quality assurance, and resources to
support a number of Ohio’s evidence-based practice initiatives. Ohio also established the Innovative Diffusion and Adoption Research Project to evaluate the implementation of these initiatives across the state.221