Predictive Analytics: The Postsecondary Use Case

(1)

Predictive Analytics: The

Postsecondary Use Case

The Association Conference

August 2, 2013

Heidi Hiemstra, Ph.D.

Associate Director, Research

[email protected]

(2)

What is Predictive Analytics?

Technology that learns from experience

(data) to predict the future behavior of

individuals in order to drive better

decisions.

- Eric Seigel, Predictive Analytics: The power to predict who will click, buy, like or die (2013: John Wiley and Sons).

(3)

Machine Learning

•

Data Warehouses and “the Cloud”

make it possible to

collect, manage and maintain massive numbers of

records.

•

Sophisticated technology platforms

provide computing

power necessary for grinding through calculations and

turning the mass of numbers into meaningful patterns.

•

Data mining uses descriptive and inferential

statistics

—moving averages, correlations, and regressions,

graph analysis, market basket analysis – to look inside

those patterns for actionable information.

•

Predictive techniques,

such as

neural networks and

(4)

Predicting Behavior:

(5)

Predicting Behavior:

Estimating Individual Risk

87% 43%

8% 62%

(6)

Predictive Analytics is not Social Science

Start with generalizable knowledge (theory driven, seeking causation) Propose hypothesis Develop method to test the hypothesis Collect and analyze data Prove/ disprove hypothesis Contribute back to generalizable knowledge

Social

Science

(7)

Predictive Analytics is not Social Science

Start by defining a business problem Gather related data (structured/ unstructured) Develop a model that predicts historical behavior Apply predictive associations from model to current individuals Assign a risk score to individuals Treat the individual differently based on risk score

Predictive

Analytics

(8)

Making Better Decisions

“Treat the individual differently based on risk score”

 Would this student do better in developmental mathematics or

in a supplemented college-level class?

 Will this student succeed in our nursing program?

(9)

(10)

Predictive Analytics Reporting (PAR) Framework

• Funded by Bill & Melinda Gates Foundation in 2011, 2012 • Managed by WICHE Cooperative for Educational

Technologies, operated by WCET core project team • 16 institutional partners

▫ 7 4-year schools

▫ 5 community colleges ▫ 4 for-profit institutions • 12.5M course level records • 1.7M student level records

• In-kind donations to date

▫ Blackboard

▫ iData

(11)

Institutional Partners

American Public University System*

Ashford University Broward College Capella University

Colorado Community College System*

Lone Star College System Penn State World Campus Rio Salado College*

Sinclair Community College

Troy University

University of Central Florida University of Hawaii System* University of Illinois

Springfield*

University of Maryland University College

University of Phoenix*

Western Governors University

(12)

Structured Data

•

Common data

definitions = reusable

predictive models and

meaningful

comparisons.

•

Openly published via a

cc license @

https://public.datacook

book.com/public/institu

tions/par

(13)

Data Inputs

Student Demographics & Descriptive Gender Race Prior Credits Perm Res Zip Code

HS Information Transfer GPA Student Type Student Course Information Course Location Subject Course Number Section Start/End Dates Initial/Final Grade Delivery Mode Instructor Status Course Credit Student Academic Progress Curent Major/CIP Earned Credential/CIP Student Financial Information

FAFSA on File – Date Pell Received/Awarded – Date Course Catalog Subject Course Number Subject Long Course Title Course Description Credit Range ** Future Lookup Tables

Credential Types Offered Course Enrollment Periods

Student Types Instructor Status Delivery Modes Grade Codes Institution Characteristics Possible Additional ** Placement Tests NSC Information SES Information Satisfaction Surveys College Readiness Surveys

(14)

Actionable Predictive Models

Gateway Course Demonstration

(15)

(16)

Student Intervention Exercise

List five student support programs or policies at

your institution (2 min):

1.

2.

3.

4.

5.

Timer Bar

(17)

PAR Student Success Matrix (SSM

x

)

Literature-based tool for benchmarking student

services and interventions

(18)

(19)

Academic Cycle Phases

Academic Career

Connection: application to enrollment

Entry: first weeks/ course/semester Progress: continuation

toward ed. obj. Completion: achieve educational objective

Course

Connection: advising to enrollment

Entry: first days/weeks, withdrawal

Progress: midterm and beyond

(20)

Intervention Focus

(21)

SSM

x

Completion

(22)

Student Intervention Exercise

At your table, work as a team to place all of the

interventions previously listed into the large

Student Success Matrix provided (10 min).



If the same intervention was listed by multiple people,

negotiate a combined response.



Only list an intervention once per row. Use arrows to

draw across the academic cycles if needed.



If the program is for all students, label it “G;” if it is for a

(23)

Food for thought…

Looking at the distribution of interventions across rows

and columns? Where are programs concentrated?

Where are there gaps?

How well does this reflect the supports you believe

your students need to succeed?

Are you measuring the effectiveness of any of these

programs?

(24)

SSM

x

Early Results



15 institutions



659 interventions contributed across predictor

Use of Aggregated Data



Benchmark dashboards

currently in development



Research on use of

predictive analytics in

postsecondary ed.



Research on the

effectiveness of

interventions



Generalizable knowledge

(27)