Using the Ohio Scales for Family Engagement, Treatment Planning, and Outcomes Tracking

(1)

Ben Ogles Ben Ogles Ohio University Ohio University with with Charlotte Williams Charlotte Williams Family Advocate Family Advocate

Using the Ohio Scales for

Family Engagement, Treatment

Planning, and Outcomes

Tracking

Thanks To

Office of Program Evaluation & Research, Office of Program Evaluation & Research, Ohio Department of Mental Health

Ohio Department of Mental Health

Southern Consortium for ChildrenSouthern Consortium for Children

Participating agenciesParticipating agencies

Participating familiesParticipating families

Multiple graduate studentsMultiple graduate students

“

If providers don

’

t measure

outcomes, someone else will.

”

(2)

Overview

Outcomes and AccountabilityOutcomes and Accountability

ODMH Consumer Outcomes System and ODMH Consumer Outcomes System and The Ohio Scales

The Ohio Scales

Using the Ohio ScalesUsing the Ohio Scales

OUTCOMES!

A Central Focus over the last 15 yearsA Central Focus over the last 15 years

Behavioral health, education, healthcareBehavioral health, education, healthcare

Books, articles, task forcesBooks, articles, task forces

Public and privatePublic and private

Administrators, providers, consumers, and Administrators, providers, consumers, and payors

payors

AN EXAMPLE FROM HEALTH CARE

“

“Programs focused on outcomes research, Programs focused on outcomes research, measurement, and management measurement, and management

have emerged over the past decade as a result of have emerged over the past decade as a result of rapidly rising healthcare costs, questions about the rapidly rising healthcare costs, questions about the

effectiveness of medical intervention, and the effectiveness of medical intervention, and the need for efficient delivery of care (

need for efficient delivery of care (MowinskiMowinski& & Staggers, 1997).

Staggers, 1997).””

(3)

Use Research Based Instruments

to Assess Outcomes in Clinical

Settings

To improve treatmentTo improve treatment

To enhance scienceTo enhance science

To provide accountabilityTo provide accountability

To be ethically responsible for qualityTo be ethically responsible for quality

(Barlow, 1982; Ogles et al., 2002) (Barlow, 1982; Ogles et al., 2002)

Ohio

’

s Public Sector Response

Core, common outcome battery for public Core, common outcome battery for public mental health services

mental health services

Implemented state-Implemented state-widewide

Move toward outcomes driven Move toward outcomes driven management

management

Reduce process measurementReduce process measurement

Review and Improve the outcomes system Review and Improve the outcomes system

ODMH Youth Outcomes

Ohio Scales (Parent)Ohio Scales (Parent)

Ohio Scales (Youth –Ohio Scales (Youth –ages 12ages 12--18)18)

(4)

Desired Characteristics

Multiple sources (parent, youth, worker)Multiple sources (parent, youth, worker)

Multiple content areas (problems, Multiple content areas (problems, functioning, satisfaction,

functioning, satisfaction, hopefulness)

hopefulness)

Practical (cost, time, clinical use)Practical (cost, time, clinical use)

Psychometrically rigorous (reliable, valid, Psychometrically rigorous (reliable, valid, sensitive to change)

sensitive to change)

Final Parent Rated Ohio Scales

Problem Severity -Problem Severity -20 Items20 Items

Functioning -Functioning -20 items20 items

Hopefulness -Hopefulness -4 items4 items

Satisfaction -Satisfaction -4 items4 items

Final Youth Rated Ohio Scales

Hopefulness -Hopefulness -4 items4 items

(5)

Final Agency Worker Rated Ohio Scales

Markers (e.g., arrests, suspensions)Markers (e.g., arrests, suspensions)

ROLES (Hawkins et al., 1992)ROLES (Hawkins et al., 1992)

Psychometric properties

Ogles, B. M., Dowell, K., Hatfield, D, Melendez, G., & Ogles, B. M., Dowell, K., Hatfield, D, Melendez, G., & Carlston

Carlston, D. (2004). The Ohio Scales. In M. E. , D. (2004). The Ohio Scales. In M. E. MaruishMaruish (Ed.),

(Ed.), The use of psychological testing for treatment The use of psychological testing for treatment

planning and outcome assessment (3rd ed., Vol. 2) (pp.

275

275--304). Hillsdale, New Jersey: Lawrence 304). Hillsdale, New Jersey: Lawrence EarlbaumEarlbaum..

OglesOgles, B. M., , B. M., MelendezMelendez, G., Davis, D. C., & , G., Davis, D. C., & LunnenLunnen, K. M. , K. M.

(2001).

(2001). The Ohio Scales: Practical Outcome AssessmentThe Ohio Scales: Practical Outcome Assessment. .

Journal of Child and Family Studies, 10

Journal of Child and Family Studies, 10, 199, 199--212.212.

Technical Manual Technical Manual -

-http://

http://oak.cats.ohiou.edu/~ogles/ostechmanual.pdfoak.cats.ohiou.edu/~ogles/ostechmanual.pdf

Administration

Protocols available at ODMH Web SiteProtocols available at ODMH Web Site

General guidelines General guidelines

–

–parents/primary caregivers and youth 12+parents/primary caregivers and youth 12+

administer prior to treatment, periodically, and terminationadminister prior to treatment, periodically, and termination

provide help with reading as necessaryprovide help with reading as necessary

–

–agency worker agency worker

following intake and periodically at the same time as parent following intake and periodically at the same time as parent and youth

and youth

See also the UserSee also the User’’s Manual s Manual -

-http://

(6)

Administration

Administration –

–

Family Engagement

Give a brief, simple description of the Give a brief, simple description of the measures

measures

Make sure they know it will be used to Make sure they know it will be used to create the intervention plan and to assess create the intervention plan and to assess progress

progress

Ensure them their point of view –Ensure them their point of view –on the on the scales and in

scales and in--person person --will inform will inform treatment

treatment

Scoring

Add up items for each of the 4 scalesAdd up items for each of the 4 scales

Missing data?Missing data?

– Ask respondent about items that are left blank – 4 or fewer blank - use Tables to get the total – If more than four blank, no total score

No reverse scored itemsNo reverse scored items

ROLES instructions in the User’ROLES instructions in the User’s Manuals Manual

See example case (George J.)See example case (George J.)

41 39 37 35 33 40 38 36 34 32 39 36 34 33 31 38 35 33 32 30 . . . . . . . . . . . . . . . 4 4 3 3 3 3 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 3 2 1 ↓

Number of Missing Items Sum of

completed items

(7)

15. Feeling lonely and having no friends 0 1 2 3 4 5

16. Feeling anxious or fearful 0 1 2 3 4 5

17. Worrying that something bad is going to happen 0 1 2 3 4 5

18. Feeling sad or depressed 0 1 2 3 4 5

19. Nightmares 0 1 2 3 4 5

20. Eating problems 0 1 2 3 4 5

(Add ratings together) Total ____

1 6 6 13

Initial Assessment

Development of Treatment PlanDevelopment of Treatment Plan

–

–Critical itemsCritical items –

–Identify target problemsIdentify target problems –

–Identify functional strengthsIdentify functional strengths –

–Charting initial scores Charting initial scores --severityseverity –

–Comparison of parent and childComparison of parent and child

All can be used to engage the familyAll can be used to engage the family

Critical Items

Parent or youth report of -Parent or youth report of

-–

–#7 drug or alcohol use#7 drug or alcohol use –

–#8 breaking the law#8 breaking the law –

–#12 hurting self#12 hurting self –

(8)

7. Using drugs or alcohol 0 1 2 3 4 5 8. Breaking rules or breaking the law (out past

curfew, stealing)

0 1 2 3 4 5

9. Skipping school or classes 0 1 2 3 4 5

10. Lying 0 1 2 3 4 5

11. Can’t seem to sit still, having too much energy 0 1 2 3 4 5

12. Hurting self (cutting or scratching self, taking pills)

0 1 2 3 4 5

13. Talking or thinking about death 0 1 2 3 4 5

Critical Items

-

Family Engagement

Dialogue with parent about any items that Dialogue with parent about any items that are endorsed.

are endorsed.

Consider how informed the parent is about Consider how informed the parent is about youth slang for these issues.

youth slang for these issues.

Consider parental stress level.Consider parental stress level.

Consider amount of time parent is with Consider amount of time parent is with the youth.

the youth.

Be sensitive in your approach to this topic.Be sensitive in your approach to this topic.

Target Problems

Identify 3 or 4 problems rated the highest Identify 3 or 4 problems rated the highest by the parent and youth.

by the parent and youth.

–

–Theme or focusTheme or focus –

–Specific target itemSpecific target item

This may help to focus an interview or This may help to focus an interview or reveal nuances of consumer perceptions reveal nuances of consumer perceptions following an interview

following an interview

(9)

Target Problems

Target Problems –

–

Family Engagement

Solicit input about issues that are not on Solicit input about issues that are not on the forms

the forms

Emphasize uniqueness of familiesEmphasize uniqueness of families

Use standardized form, but allow families Use standardized form, but allow families to rank issues in terms of what is critical to rank issues in terms of what is critical to them.

to them.

Functional Strengths

Identify functioning items rated as 3 or 4Identify functioning items rated as 3 or 4

Identify problem items rated 0Identify problem items rated 0

May help to locate strengths that can May help to locate strengths that can facilitate change.

facilitate change.

Useful for initial treatment planningUseful for initial treatment planning

Can incorporate resilience researchCan incorporate resilience research

Resilience

““A set of qualities that foster a process of A set of qualities that foster a process of

successful adaptation and transformation successful adaptation and transformation despite risk and adversity

despite risk and adversity””(Benard(Benard, 1995), 1995)

““Protective buffersProtective buffers……appear to make a more appear to make a more

profound impact on the life course of individuals profound impact on the life course of individuals who grow up and overcome adversity than do who grow up and overcome adversity than do specific risk factors

(10)

Resilience Qualities of the Child

Social CompetenceSocial Competence––responsiveness, responsiveness,

flexibility, empathy, communication skills, flexibility, empathy, communication skills, humor.

humor.

Problem Solving SkillsProblem Solving Skills––ability to plan, ability to plan,

resourceful, creative thinking (reflective) resourceful, creative thinking (reflective)

Critical ConsciousnessCritical Consciousness––reflective awareness reflective awareness

of oppression and creative overcoming of oppression and creative overcoming

AutonomyAutonomy––independence, task mastery, independence, task mastery,

control, self control, self--efficacyefficacy

Sense of purposeSense of purpose––optimism, persistence, optimism, persistence,

hopefulness, goal directed

hopefulness, goal directed ((BenardBenard, 1995), 1995)

Some External Factors that

Contribute to Resilience

Good intellectual functioningGood intellectual functioning

Economic resourcesEconomic resources

Caring RelationshipsCaring Relationships

High Expectations (authoritative High Expectations (authoritative parenting)

parenting)

Opportunities to Participate (school, Opportunities to Participate (school, prosocial

prosocialorganizations)organizations) (

(MastenMasten& Coatsworth& Coatsworth, 1998), 1998)

Example: Some Ohio Scales Items

that provide information

about Resilience

Social competence Social competence --> Functioning items 1> Functioning items 1--4, 204, 20

Autonomy Autonomy --> Functioning items 14, 18, 19> Functioning items 14, 18, 19

Sense of purpose Sense of purpose --> Hopefulness Scale> Hopefulness Scale Clinicians could identify others Clinicians could identify others

based on their experience based on their experience

(11)

Strengths

–

Family Engagement

Recognize things have been difficult Recognize things have been difficult leading to treatment and the parent may leading to treatment and the parent may have trouble identifying strengths

have trouble identifying strengths

Help them to find strengths in what they Help them to find strengths in what they may consider to be routine

may consider to be routine

Lack of opportunity may prevent some Lack of opportunity may prevent some abilities to surface

abilities to surface

Initial Severity

Chart scores using figures in the User’Chart scores using figures in the User’s s Manual (reproduce as needed).

Manual (reproduce as needed).

Clinical cutoff Clinical cutoff

–

–Problem Severity 20Problem Severity 20 –

–Functioning 51 (60 for youth)Functioning 51 (60 for youth)

Internalizing/Externalizing PatternInternalizing/Externalizing Pattern

Level of Severity via Calibration

Level of

Severity

Severity CAFASCAFAS Ohio Scales Ohio Scales FunctioningFunctioning

Ohio Scales Ohio Scales Problem Problem Severity Severity 0 0 --nonenone 00--1010 66+66+ 00--99 1 1 --mildmild 2020--4040 5151--6565 1010--1919 2 2 --moderatemoderate 5050--9090 3535--5050 2020--3636 3 3 --severesevere 100100--130130 2323--3434 3737--5252 4 4 --extremeextreme 140+140+ < 23< 23 53+53+

(12)

Initial Severity

Initial Severity –

–

Family Engagement

Remind parent the total score comes from Remind parent the total score comes from their ratings of their child (Validate) their ratings of their child (Validate)

Verify that their perception matches the Verify that their perception matches the total score level of severity

total score level of severity

Work with the parent to identify Work with the parent to identify appropriate types and intensities of appropriate types and intensities of treatment

treatment

Compare Sources

Compare parent and youthCompare parent and youth

–

–Internalizing Internalizing ––youth tend to rate higheryouth tend to rate higher –

–Externalizing Externalizing ––parents tend to rate higherparents tend to rate higher

Compare parent and youth with agency Compare parent and youth with agency worker

worker

–

–Situational issues (court referral)Situational issues (court referral)

Parent-Youth Difference 70 .0 60 .0 50_.0 40 .0 30 .0 20_.0 10 .0 0. 0 -1 0.0 -20 .0 -3 0.0 -4 0.0 -50 .0 -6 0.0

Discrepancy on Problem Severity

# o f c a s e s 500 400 300 200 100 0 Std. Dev = 14.98 Mean = 2.6 N = 2153.00

(13)

Factors associated with Agreement

Gender (better total agreement with daughters Gender (better total agreement with daughters

than sons; more item agreement with sons) than sons; more item agreement with sons)

Rater Relationship (parents > others)Rater Relationship (parents > others)

Ethnicity (Hispanic > Caucasian,Ethnicity (Hispanic > Caucasian,

multi

multi--ethnic > African American) ethnic > African American)

Area of assessment (internalizing > externalizing Area of assessment (internalizing > externalizing

& conduct ) & conduct )

Diagnosis (internalizing > externalizing;Diagnosis (internalizing > externalizing;

reverse at item level) reverse at item level)

(

(CarlstonCarlston, 2003), 2003)

Discrepancy and Change

Bigger discrepancy -Bigger discrepancy -> increased dropout> increased dropout

Bigger discrepancy (with parent rating Bigger discrepancy (with parent rating higher)

higher) --> poorer outcome> poorer outcome

(

(CarlstonCarlston, 2003), 2003)

Comparing Scores

Comparing Scores ––Family EngagementFamily Engagement

Dialogue with parent to pin point the Dialogue with parent to pin point the reasons for large discrepancies. reasons for large discrepancies.

This may also be an opportunity to gather This may also be an opportunity to gather information about things the parent has information about things the parent has tried in the past (including past treatment) tried in the past (including past treatment)

(14)

Use Initial Data to Inform the In

-

-Person Clinical Assessment

Person Clinical Assessment

Look at critical items, strengths, problems, Look at critical items, strengths, problems, level of severity, discrepancy,

level of severity, discrepancy, internalizing

internalizing--externalizing pattern to externalizing pattern to inform the initial assessment through: inform the initial assessment through:

–

–Collateral contacts, Collateral contacts, –

–Interview with child, Interview with child, –

–Meeting with family, Meeting with family, –

–etc.etc.

Using Initial Data to Empower the

Family in Subsequent Contacts

Verify parent and youth report in faceVerify parent and youth report in face--toto--face face

contacts contacts

Expand on strengthsExpand on strengths

Clarify goals and focus of servicesClarify goals and focus of services

Consider the level of parent and youth hopeConsider the level of parent and youth hope

Describe the use of scales for tracking change Describe the use of scales for tracking change

and informing the family and informing the family

Discuss differences in parent and youth ratingsDiscuss differences in parent and youth ratings

Rate Worker Scale

Once you have sufficient information to Once you have sufficient information to begin an individual behavioral plan, then begin an individual behavioral plan, then rate the worker version of the scale. rate the worker version of the scale.

The worker rating should not be a rote The worker rating should not be a rote replication of the parent form. It should replication of the parent form. It should be informed by, but independent of, be informed by, but independent of, family ratings.

(15)

Review the George J.

Summary Form

Summary form is available to guide the Summary form is available to guide the use of the Ohio Scales in the initial use of the Ohio Scales in the initial treatment planning process

treatment planning process

See HandoutSee Handout

Practice Case

Roberta R.Roberta R.

See HandoutsSee Handouts

Section Summary

Outcome is here to stayOutcome is here to stay

Ohio has a core battery that will help for Ohio has a core battery that will help for children with mental health needs children with mental health needs ––it is it is not perfect, but it is a good start

not perfect, but it is a good start

The data can be used for the initial The data can be used for the initial assessment to inform treatment planning assessment to inform treatment planning and for family engagement and

and for family engagement and empowerment

(16)

Using the Ohio Scales to

Improve Quality

Individual clinicians develop tracking Individual clinicians develop tracking protocols for use with each client protocols for use with each client

Supervision of the individual clinician with Supervision of the individual clinician with the individual client

the individual client

Aggregate data for management and Aggregate data for management and marketing marketing

Tracking Change

(Outcome monitoring)

Items Items

Total scoresTotal scores

Compare scales (problems/functioning)Compare scales (problems/functioning)

Clinical significanceClinical significance

– –Recovery Recovery – –ImprovementImprovement

Individual Items

Can focus on target problemsCan focus on target problems

Simulates the “Simulates the “Target ComplaintsTarget Complaints”” measures used in many outcome studies measures used in many outcome studies

Example using the Treatment for Example using the Treatment for Depression Collaborative Research Depression Collaborative Research Program (TDCRP) study

(17)

Item Change

–

Family Engagement

Explain that tracking scores on an item is Explain that tracking scores on an item is like tracking daily blood pressure or blood like tracking daily blood pressure or blood sugar levels and that it can be used to see sugar levels and that it can be used to see progress.

progress.

Gives parents a frame of reference and Gives parents a frame of reference and helps them to be more attentive to the helps them to be more attentive to the things that are being tracked.

things that are being tracked.

Total Scores

Primary use of the scales to track Primary use of the scales to track individual change using the total scores individual change using the total scores

Plot total problem severity and functioning Plot total problem severity and functioning over time to monitor and evaluate

over time to monitor and evaluate progress

progress

Total Score Change

Total Score Change –

–

Family Engagement

Share the change information with the Share the change information with the family.

family.

Provide a simple explanation of the graph.Provide a simple explanation of the graph.

Verify parent perspective of change is Verify parent perspective of change is consistent with the graph.

(18)

Comparison of scales

Theories of changeTheories of change

Howard’Howard’s phase model of therapys phase model of therapy

–

–remoralizationremoralization(hope)(hope) –

–remediation (problems/symptoms)remediation (problems/symptoms) –

–rehabilitation (functioning)rehabilitation (functioning)

Scale Change

–

Family Engagement

Be realistic and help the family to be Be realistic and help the family to be realistic about what can be accomplished realistic about what can be accomplished

At the same time, let the family define At the same time, let the family define their success.

their success.

Clinical Significance

Improvement (amount of change)Improvement (amount of change)

Recovery (end point of change)Recovery (end point of change)

Both -Both -Clinical SignificanceClinical Significance

(Jacobson &

(19)

Movement into the Functional

Distribution

Graphic Representation

0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Pre-treatment Problem Severity Post-treatment Problem Severity *A *B RELIABLE IMPROVEMENT RELIABLE DETERIORATION CLINICAL SIGNIFICANCE *E *D *C

Clinical Change

Clinical Change –

–

Family Engagement

Very Technical –Very Technical –find a way to explain in find a way to explain in simple terms

simple terms

Don’Don’t forget that what is meaningful t forget that what is meaningful change to the parent may not match with change to the parent may not match with the statistical definition.

(20)

Use Outcome Data to Empower the

Family in Subsequent Contacts

Verify parent and youth outcome report in faceVerify parent and youth outcome report in face-

-to

to--face contacts.face contacts.

Use progress to leverage additional progress.Use progress to leverage additional progress.

Clarify goals and focus of services when Clarify goals and focus of services when

additional progress is needed. additional progress is needed.

Consider altering treatment in collaboration with Consider altering treatment in collaboration with

the family when encountering deterioration. the family when encountering deterioration.

Help identify when treatment might end.Help identify when treatment might end.

Maintain hope.Maintain hope.

DonDon’’t have to wait to ret have to wait to re--evaluate or change the evaluate or change the

plan plan

Satisfaction

-

Family Engagement

Remember to visit with the family about Remember to visit with the family about their satisfaction with treatment and their their satisfaction with treatment and their perception about inclusion in treatment perception about inclusion in treatment planning.

planning.

Services are not dependent on feedback.Services are not dependent on feedback.

Watch for changes in satisfaction.Watch for changes in satisfaction.

Review the George J. Summary

Form

Summary form to provide protocol for Summary form to provide protocol for using the Ohio Scales to monitor change using the Ohio Scales to monitor change

(21)

Case Roberta R.

Complete the form using the caseComplete the form using the case

Section Summary

Outcome implementation provides Outcome implementation provides opportunities to move beyond

opportunities to move beyond measuringmeasuring to

to monitoringmonitoringor even managingor even managingthe the quality of mental health services. quality of mental health services.

Outcome data can be integrated into Outcome data can be integrated into routine clinical practice for tracking routine clinical practice for tracking change client

change client--byby--client. client.

Outcome Management

Clinical SupervisionClinical Supervision

Aggregate AnalysesAggregate Analyses

–

–SatisfactionSatisfaction –

–Client Characteristics, treatment factorsClient Characteristics, treatment factors –

–Reporting/Marketing/AccountabilityReporting/Marketing/Accountability –

(22)

Clinical Supervision

–

The Missing Piece

Client outcome data is rarely considered Client outcome data is rarely considered when studying supervision (usual focus is when studying supervision (usual focus is on the supervisee/trainee).

on the supervisee/trainee).

Client outcome data (other than informal Client outcome data (other than informal testimonials or verbal report) is rarely testimonials or verbal report) is rarely included in the process of clinical included in the process of clinical supervision.

supervision.

“The impact of clinical supervision on “The impact of clinical supervision on client outcome is considered by many to client outcome is considered by many to be the acid test of the efficacy of be the acid test of the efficacy of supervision

supervision””(Ellis & (Ellis & LadanyLadany, 1997, p. 485), 1997, p. 485)

““. . there is still virtually no attention . . there is still virtually no attention given to client change . . in this line of given to client change . . in this line of research

research””(Watkins, 1995, p. 668)(Watkins, 1995, p. 668)

Using Outcome Data as a

Tool in Supervision

Case AssignmentCase Assignment

Treatment Planning/family engagementTreatment Planning/family engagement

Periodic review of casesPeriodic review of cases

(23)

Example for Supervision

-

-Prioritizing cases

Prioritizing cases

Outcome data can be used to select cases Outcome data can be used to select cases for supervision review

for supervision review

Aggregate Example 1Aggregate Example 1

Review the graphs and select the 3 cases Review the graphs and select the 3 cases you would want to review first in a time you would want to review first in a time- -limited supervision session.

limited supervision session.

Supervision follow

-

up

More detailed information available for More detailed information available for using ODMH consumer outcomes in using ODMH consumer outcomes in supervision is available at the ODMH web supervision is available at the ODMH web site as part of the Outcomes Tool kit. site as part of the Outcomes Tool kit.

Aggregate

-

Satisfaction

Can examine the four satisfaction items Can examine the four satisfaction items individually or in aggregate form. individually or in aggregate form.

Does not provide the comprehensive Does not provide the comprehensive information many agency satisfaction information many agency satisfaction surveys collect (only 4 items)

surveys collect (only 4 items)

Global, relevant, and from many Global, relevant, and from many consumers

consumers

(24)

Aggregate

-

Clinical

Average totals across an agency, program, Average totals across an agency, program, or clinician to report rates of change. or clinician to report rates of change.

Compare groups of consumers (e.g., by Compare groups of consumers (e.g., by residence, service, referral source). residence, service, referral source).

Research oriented or administratively Research oriented or administratively oriented questions.

oriented questions.

Aggregate

-

Reporting

Outcome reports can be used for Outcome reports can be used for marketing or accountability purposes marketing or accountability purposes (e.g., (e.g., “

“creating public valuecreating public value””Moore, 1995)Moore, 1995)

Aggregate

-

Provider Profiling

Can also aggregate data by Can also aggregate data by provider/clinician

provider/clinician

–

–Useful for supervision and trainingUseful for supervision and training –

(25)

Example 5

–

Provider profiling

0 5 10 15 20 25 30 35 40 45 50 % R ec ov er ed 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Number of Sessions Training Clinic Dr. L. J.

(Lambert, Okiishi, Finch, & Johnson, 1998)

Example 6

–

Comparing therapists

0 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 Whole Center Therapist #16 Therapist #11

(Okiishi, Lambert, & Ogles, 2003).

Section Summary

Outcome data may also be useful for Outcome data may also be useful for clinical supervision.

clinical supervision.

Aggregate data may be used to identify Aggregate data may be used to identify client characteristics, programs, or client characteristics, programs, or providers that may need focus. providers that may need focus.

(26)