Using the Power of Predictive Analytics for Case Outcomes

(1)

ERICSA 52nd_{Annual Training Conference & Exposition ▪ April 26 – 30} Hershey Lodge ▪ Hershey, Pennsylvania

Using the Power of Predictive

Analytics for Case Outcomes

(2)

ERICSA 52nd Annual Training Conference & Exposition ▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Participants



Presenter – Edward V. Lehman, Jr.



Director, Case Processing & Data Management



Philadelphia Family Court – Domestic Relations

Division



Presenter – Steven J. Golightly, Ph.D.



Los Angeles County Child Support



Moderator – Joyce Match



Business Analyst Manager



Pennsylvania Bureau of Child Support

(3)

ERICSA 52nd_{Annual Training Conference & Exposition}▪ April 26 – 30

Hershey Lodge ▪ Hershey, Pennsylvania

Predictive

(4)

ERICSA 52nd_{Annual Training Conference & Exposition}▪ April 26 – 30 ▪ Hershey Lodge ▪ Hershey, Pennsylvania

Access to the right information is extremely important

What is the requirement?

Increase

Visible

Results

Reduce

Information

Overload

Work the

Right Cases

Effective

Case Worker

Management

Determine Next

Appropriate

Action

Proactively

(5)

Understanding Past, Measuring Present,

Predicting the Future

(6)

Predictive Analytics in Child Support

• Better estimates of delinquencies, child support collections

• Fewer child support cases in arrears

• Model results can be used to quantify historical process inefficiencies

• Significant marketing value – “We Know Our Citizens”

• More reliable payments for children

• Shift from reactive enforcement to early intervention

• Learning component for new case management approaches

• Assign high risk cases to case workers sooner

• Different approaches for different regions and different case types

• Automate certain research processes

• Reduce user guesswork and “cherry picking”

• Prioritize workload based on high impact

• Incorporate historical experience to drive future activities

• Eliminate a one size fits all approach

• Efficiency of customer service interactions

• Over time reduce future workload

Improved

Workforce

Effectiveness

Increased

Resource

Allocation

Efficiency

Improved

Outcomes

(7)

Predictive Modeling for Child Support

o

Child support enforcement has traditionally been a reactive process.

o

What if we had a tool that could help us predict which NCPs are

most likely to become in-arrears in the near future?

o

We could use such a tool to:

o

Prevent arrears

o

Decrease custodial parent complaints

o

Take the right action on a case at the right time

o

Assign the right case workers to the right cases

o

Gather information for potential policy changes

(8)

How do Models Work?



Statistical models just refine what we do naturally all

the time.



For example: which of these NCP’s is most/least

likely to pay next month?

Gene:

35 years old

$2000 in arrears

No other children on case

No Salary attachment

History of family violence

Case is a IV-A assistance

Tom:

28 years old

$100 in arrears

1 other child on case

Salary attachment

No history of violence

Case is not an

assistance case

Rick:

39 years old

$8000 in arrears

No other children on case

No salary attachment

No history of violence

NCP has other child support

cases

(9)

(10)

Pennsylvania’s Predictive Analytics solution included

updates to the existing Performance Improvement Module

(PIM) solution and a new application to calculate a predictive

analytics “score.”

• Allows a worker to calculate a score for a case based on 20

variables

• Score is the likelihood of the defendant to pay 80% towards

current support obligation in the next three months

• Information required to generate the score is easily accessed

from the system at the time the support order is created or

modified

• Provides a list of upcoming child support establishment

conferences with the cases assigned to the worker to allow the

score creation prior to the conference

.

(11)

PIM/Predictive Analytics

o

Targeted case lists

o

Suggested case actions

o

Predictive score and

reasons

o

Performance metrics

o

Projects for targeted,

custom outreach including

text messages

(12)

Payment Score Calculator Objectives

• Establish a good

payment pattern:

Increased quantity

and frequency of

collections

• More effective

meetings with

defendants

• Make available new

methods of reaching

out

• Identifying

opportunities for

proactive

enforcement activities

• Maximize

performance metrics

• Minimize costs

related to

enforcement

• Improve

cost/effectiveness

ratios

• Effective case

assignment based on

scores

• Establishing consistent

payment patterns

based on successful

business process

actions for various

scores

• Provide the right

services at the right

time to encourage

compliance with the

order

• Let the defendant know

the case is being

(13)

Variable Selection

Final model contains

20 variables

36 variables considered for

modeling after Exploratory Data

Analysis [EDA] Phase

64 candidate variables

created during data

scrubbing / brainstorming

phase

Over

400 data elements

collected and considered in

variable creation phase

The case for which a score is being calculated is compared against the

historical behavior of thousands of cases with similar characteristics

(14)

Example Predictive Variables

Number

Predictive Variables

1 Collection Indicator

2 High Number of Enforcement Activities Indicator

3 Number of Cases

4 Number of Enforcement Activities

5 Balance of Arrears

6 Defendant Net Income

7 Active Income Attachment Indicator

8 Number of Defendant Member Addresses on MADD

9 Number of Defendant Employers

10 Distance between the Defendant and Plaintiff

These are just a few examples of predictive variables used in

Pennsylvania. Variables used in each State are different.

(15)

Predictive Modeling Scores

Probability of NCP paying 80 percent on current support in

the three months after support order issued/modified.

0 – 30%

31 – 50%

(16)

Confidence in the Score

Payment

Score

Average Percent of Current

Support

1 51%

2 66%

3 77%

4 87%

To validate the accuracy of the payment score, a study of 5,000 PACSES cases was completed

For each payment score, the average FYTD percent of current support collected was calculated.

A direct relationship between payment score and percent of current paid was found (i.e., the

higher the payment score the higher the percent of current paid.

(17)

Planned enhancements to the existing functionality of the

Payment Score Calculator

• Development of a mechanism for calculating a Payment

Score for all open cases

• Quarterly refresh the Payment Score Calculations on all

open cases

• Allow worker to complete a guideline calculation if the

PSC has been updated within the last three months

• Modification of PIM to add several pre-defined filters

which use the Payment Score as one of the factors for

consideration

• Implementation planned for June, 2015

PSC Enhancements

(18)

After PIM was implemented, Pennsylvania became the first State to achieve

above 80% in both percent of current and percent of arrears

PIM/Predictive Analytics Effect in Pennsylvania

PA has been ranked #1 in the United States for both percent of current

and percent of arrears, both of which have increased significantly since

(19)



Revenue Impact: Exceeding 80% of collections targets



Accuracy: “Likely payers” 4x more likely to be above 50% paid than “very unlikely payers”



Timeliness: Allows taking action at time of support order, before payments are missed



Actionability: Caseworker gets clear guidance on recommended steps



Policy Insights: Ability to better segment NCPs to better understand policy impacts



Workforce effectiveness:

•

Shift from reactive enforcement to pro-active education

•

Ability to prioritize workload based on high impact



Efficiency: Increased effectiveness of retention outreach campaign, audits



Time to production: Scored 17,000 cases within first three months



Transparency: Unified view across all regions and business processes



Marketing value: Helps build positive relationships with our clients. Everyone wins when

payments are made

(20)

Edward V. Lehmann, Jr.

Director, Case Processing and Data Management

Philadelphia Family Court

Domestic Relations Section

[email protected]

Feel free to contact us.

(21)

ERICSA 52nd_{Annual Training Conference & Exposition ▪ April 26 – 30} Hershey Lodge ▪ Hershey, Pennsylvania

(22)

WHERE DO WE START?

Starting with the Why

Simon Sinek’s Golden Circle

(23)

WHY

(24)

TAILORING SERVICES USING

PREDICTIVE ANALYTICS

4

(25)

PREDICTIVE ANALYTICS – CLASSIFICATION MODEL

Forecast (predict) an outcome using multiple predictors

TARGET: Predict which cases we can expect a payment of at

least 70% of Current Support Amount in subsequent month

Built a Classification Model

• started with 100+ programmatic, demographic, and economic

variables

• narrowed down to 20 statistically significant variables

5 Model accurately

predicts target

variable for

90% of cases.

(26)

CASE SCORING

6 0%

10%

20%

30%

40%

50%

0

1

2

3

4

5

6

7

8

9

Payment Status (Prior Month) Prediction

(Next Month) Confidence Score

Score (Propensity Score)

1 Non-Payer

Non-Payer

.91

1

2 Non-Payer

Non-Payer

.73

3

3 Payer

Payer

.91

9

4 Payer

Payer

.73

7

5 Payer

Non-Payer

.55

5

6 Non-Payer

Payer

.55

5

(27)

PREDICTIVE ANALYTICS - SEGMENTATION

Built a Segmentation Model

• Started with same

variables as in

classification model

• Clustering Analysis

identified natural groupings

based on payment history

Group cases using one or more input fields

7 TARGET: Create cluster of cases based on case variables to

identify natural grouping of cases

(28)

EARLY SUCCESSES

Arrears Only Project - January 2014

9,486 cases with an Arrears balance, no minor children, no current

support, and no payment within last federal fiscal year

Strategy – Focused Caseload, Locate Experts

Results – $2 million in 12 months!

8 47.96%

55.99%

Department

Arrears Only

(29)

PREDICTIVE ANALYTICS FINDINGS

Arrears Only Project Success

• Personalized the client’s experience

• Case Worker a subject matter expert for

case type

9 • Predictive Analytic Model identifies cases that

perform similarly

• PA Model predicts, with a high level of confidence,

how a case will perform

• PA Model identifies characteristics and motivations

of different customer types

(30)

HOW

CREATE NEW BUSINESS MODEL

Case Ownership

Case Segmentation

EST Zero Order Non-Paying Occasion al Paying Arrears Only Other Estab. Zero Order Paying Occasional Non-Paying Arrears Only IGR

10

(31)

SEGMENTATION MODEL

Intake

Establishment

Arrears Only

IGR

Customer

Response

11 ENFORCEMENT

(32)

ENFORCEMENT CASELOAD

Definition –

Case has an order for $0

OR

Case has a reserved or

pending supplemental

order

OR

Case has a Medical Only

order

OR

Case does not have an

active order, but case

has an arrears balance

and child is a minor

Occasional

Paying

Zero Order

Non-Paying

12 ZERO ORDER

QUADRANT

(33)

ENFORCEMENT CASELOAD

Definition –

Case has an order

AND

Current Support Amount

is a dollar amount greater

than $0

AND

NCP pays regularly

AND

NCP pays over 70% of

CS amount

Occasional

Paying

Zero Order

Non-Paying

13

(34)

ENFORCEMENT CASELOAD

Definition –

Case has an order

AND

Current Support

Amount is a dollar

amount greater than

$0

AND

NCP pays

irregularly

OR

NCP pays less than

70% of CS amount

Occasional

Paying

Zero Order

Non-Paying

14 OCCASIONAL PAY

QUADRANT

(35)

ENFORCEMENT CASELOAD

Definition –

Case has an order

AND

Current Support

Amount is a dollar

amount greater than

$0

AND

NCP has not made a

payment during the

current Federal Fiscal

Year

Occasional

Paying

Zero Order

Non-Paying

15

(36)

Occasional

 Send letters of Introduction to new case manager

 UIB/DIB/ Mod review (CMT Sort)

 Review for SLMS suspension – Manual submission for NCP’s with valid license

 Incarcerated Special Mod/closing Project

 CMT High TMSO paying less than 90% sort

 Review EFO case function cases for order for subsequent child

 Work LC005 task for IWOs

 Review CMT for SSI benefits

Paying

 Low Order with reported Earnings Mod Project

 Admin IWO Project

 Incarcerated Special Mod Project

 CMT High TMSO paying less than 90% sort

 UIB/DIB Mod review (CMT Sort)

 New Order clerical contact of NCP (EI)

 Work LC005 task for IWOs

Zero

Order

 Zero Mod Project

 Review Paid in Full cases to confirm emancipated DP’s updated for auto closures.