• No results found

Aaron, can you do a presentation on data?


Academic year: 2021

Share "Aaron, can you do a presentation on data?"

Show more ( Page)

Full text


Action Research:

Data Collection and Evaluation Strategies

for Healing to Wellness Courts

Aaron Arnold

Director, Tribal Justice Exchange Center for Court Innovation

“Aaron, can you do a

presentation on data?”


Major Themes

Data helps you

improve your


Data collection and

analysis should be


You don’t need to

be an expert—the

entire HTWC team

should be involved.

Data helps generate

support from tribal

leaders, community,

and grant funders.


What is Data?

• Something that can be observed/measured • Does not require additional analysis

Factual information

• # of HTWC participants • # of positive drug tests • # of days spent in jail

Can be quantitative…

• types of incentives and sanctions used • type of treatment ordered

• employment history

…or qualitative

• Are we serving our target population?


Why Collect Data?

Key Component #8

• Monitoring and evaluation measure the

achievement of program goals and gauge


• Improve the HTWC process

• Oversee participant progress

• Inform funders

Center for Court Innovation 5


We Do It With…

Center for Court Innovation 7

Provides immediate and useful feedback about everyday program operations and performance

Evaluates not just whether a HTWC is working, but how it’s working, why it’s working, for whom it’s working, and how it can improve

Action Research

Action Research Questions:

Is the HTWC meeting its volume projections? What is the profile of HTWC participants? What is the course of treatment and recovery?

What are the core outcomes of our HTWC? Which participants succeed?

How can this information

be used to improve our


Developing an

Action Research Plan

1. Identify program goals

• Why was your HTWC established? What did it set out to do in the first place?

• Examples of program goals: • Reduce recidivism

• Rehabilitate addicted offenders • Process cases more efficiently

• Make courts more therapeutic/less punitive

Center for Court Innovation 9

Developing an

Action Research Plan

2. Identify program objectives

• More specific than goals (how will goals be achieved?) • Follow the SMART approach

• Specific • Measurable • Achievable • Results-oriented • Time-bound

Examples of program objectives: • Enroll 25 new participants per year • Move cases from arrest to HTWC

enrollment within one week • Achieve a 50 percent graduation rate • Reduce recidivism by 20 percent


Developing an

Action Research Plan

3. Plan to measure the objectives

• What data is needed to measure the objectives?

• See handout for examples

• Who has the data we need?

• How will this data be collected and by whom?

• Who is responsible for which data reporting tasks?

• How often should reports be updated?


Developing an

Action Research Plan

4. Other questions of interest

• What other information would be helpful to know about the HTWC that may not fall neatly under the objectives?

• What data is needed to answer these other questions? • Examples of other questions:

• Do certain categories of participants have special needs? • How long do participants take to reach key milestones? • Which sanctions and rewards are used and how often?


Developing an

Action Research Plan

5. Make changes

• Brainstorm examples of practices or policies that you might rethink depending on the results of action research? • Examples of research leading to changes:

• Special services for female clients • English-as-a-Second Language program • Adjusting initial screening procedures • Reallocation of staff time/duties

Center for Court Innovation 13

Data Collection 101

Distinguish participants from


• Acceptance rate

• Reasons for non-entry in HTWC

• Number of people actually served

• Basis for most other data analysis (e.g., retention

rate, graduation rate, recidivism)


Data Collection 101

Track current program status of all


• Open (still active in program)

• Warranted (absconded/temporarily disappeared) • Graduated (successfully completed the program) • Failed (dropped out or was terminated from program) • Incomplete


Data Collection 101

Record key dates

• Arrest date/probation violation date

• Intake date

• Participation date (or ineligibility date)

• Exit date


Key Performance Measures

• Is our HTWC screening and enrolling enough participants? • If not, diagnose the reasons why:

• Low referrals (arrests are down, judges not sending people, lack of formal referrals procedures) • Low acceptance rate (defendant

refusals, ineligibility)



Key Performance Measures

• Immediacy is critically important

• Goal should be to get

participants into treatment as

quickly as possible

• Minimize early failure (e.g.,

create “pre-placement” groups

while participants wait for

treatment beds)

Initial case



Key Performance Measures

• Longer retention in treatment leads to less long-term drug use and criminal re-offending

• Cheap and easy indicator of program success

• Aim to retain participants for at least 12 months

• National average is around 60% one-year retention





Center for Court Innovation 19

Key Performance Measures

• Can indicate whether the

HTWC’s graduation

requirement are too easy or

too hard

• National study found that

graduates spend about 15

months in the program

Time to



Key Performance Measures

• Demographics, drug use, criminal history, co-occurring disorders

• Not really a “performance” indicator, but very useful

• Identify need for additional services

• Compare with other HTWCs • Determine whether participants

match target population




Key Performance Measures

• How common are relapses and other types of noncompliance? • How frequently are participants

sanctioned and what kinds of sanctions are most often imposed? • Are certain categories of

participants more likely to fail? • Are there critical warning signs of





Use of Surveys

Action research isn’t just about “hard” data.

• Survey participants

• Feedback about treatment programs

• Understanding consequences of graduation or failure • Meeting civil legal needs

• Survey case managers

• Need for additional services • Quality of existing services


Whew! Glad That’s Done.


Goals of Data Management

Data Management Goals

Enhance case processing Make better-informed decisions Improve accountability Promote collaboration Make improvements Generate support for the HTWC 25

Enhance Case Processing

Conventional Case Management System Stands


Step 1: Assessment Interview: Substance Abuse/Treatment History

Enhance Case Processing

Screen Shot: New York State Justice Center Application (JCA) (web-based example)

Step 1: Assessment Interview: Mental Health History


Step 1: Assessment Interview: LSI-R Criminal History

Informed Decision-Making

Screen Shot: LSI-R Standardized Assessment Interview in New York State Universal Treatment Application (UTA)

Enhance Case Processing


Step 6: Litigant Compliance: Reviewing Drug Test History

Litigant Accountability

Screen Shot: New York State Universal Treatment Application (UTA)

Treatment Provider Enters Drug Test Results via Web MIS


Litigant Accountability

Comprehensive Progress Report

Screen Shot: Buffalo, New York Drug Court MIS (MS Access)

Tracking Results

Front-End Reporting Tools for Management, Grant Reporting, and Long-Term Evaluation Needs

Primary Drug

CRACK 52% ETOH 33% OPIOID 10% THC 5%


Technical Considerations


Major Themes

Data helps you

improve your


Data collection and

analysis should be


You don’t need to

be an expert—the

entire HTWC team

Data helps generate

support from tribal

leaders, community,


Training and

Technical Assistance

Center for Court Innovation


Aaron Arnold

Director, Tribal Justice Exchange

Center for Court Innovation

Phone: (315) 266-4331




Related documents

There exist many public providers that offer various on-demand virtual machines with various types of resources (e.g., AmazonEC2 offers about 25 different types of instances). In

We will also show that, whatever the quality of MTCD tools, because some uncertainties like wind prediction errors are unavoidable, even a perfect MTCD will always detect

FX Fleet Manager gives you better control and management of your field operations and allows you to make decisions that will increase profitability, reduce service costs,

To give your business users reliable data when and where they need it to make rapid and informed decisions, you need to integrate, transform, improve, and deliver trusted data

This course introduces students to the study of communication. It is designed to provide a critical overview - or survey - of the main themes and issues in the field of

Saids’ framework of Orientalism relates to this thesis’ main research question: “To what extent does a neo-Orientalist framing of ‘Palestinian terrorists’ occur in the

To really understand the key concerns, objectives, and reporting needs of other department managers, IT directors would be well advised to sit down and talk to the

Printing the solution data set shows the final values of the objective function and the constraints in Figure 3.. The objective function is defined as the sum of the

This  orchestral  piece  explores  ideas  of  a  never‐ending  cycle  of  birth,  death  and  rebirth.  In  Sanskrit  this  is  described  as  samsara.  Composed 

The undersigned specifically certifies that: (1) all statements made in this application are made for the purpose of obtaining the assistance indicated herein and are true

If conditional approval of a wetland and riparian zone conservation tax credit is granted, the application must be resubmitted to the Commission after the qualified real property

Click Add and enter the following information for each Dampening Profile that you want to configure, select Enable , and click OK :8. • Profile Name —Enter a name to identify

2) Reviewing and categorizing contemporary research of intra-vehicle IDSs (42 works) based on their detection technique, and examining each work based on the used features and

The core of our analysis are the estimates on sibling di¤erences that use average weekly working hours when the child is aged 0-3 to measure parental time inputs: given their

To help children, we created SAFEhaven, a program taught in area schools and designed to instruct children that an NBU vehicle and crew is a SAFEhaven where anyone can get help

In Panel A we report the results of CAPM and Fama-French regression of portfolio returns for return of portfolios of mutual funds managers, matched sample, and difference between

These tools can be customized to improve internal controls and promote accountability; to provide real-time status and increase visibility for management; to enhance communication

Low value added by investment advice for wealthy investors does not seem to justify the (hefty) fees. There are several potential criticisms concerning the validity of our

• Convince the authorities to take the decision to go for quality labelling and certification • Create the legal framework. • Accreditate a

To enhance selecting officials’ ability to make better informed and timely hiring decisions, a State agency (“Hiring Agency”) may review/copy specified and relevant information

The objectives of this study were: (1) to characterize genetic structure of the most popular sorghum varieties from two different agroecosystems, the Lake Victoria basin region

To do that, we provide comprehensive health plans, health resources, and wellness programs that empower you to make informed decisions and support your overall health...

We will seek to better support consumers to help ensure they can make decisions about their food in an informed manner, and to enable them to influence