Northwestern University Feinberg School of Medicine
Mastery Learning of Clinical Skills
2012 ICRE Conference
William C. McGaghie, PhD
Diane B. Wayne, MD
Objectives
•
Recognize variation in clinical skill among
experienced professionals
•
Review principles of mastery learning and
deliberate practice
•
Describe advantages of mastery learning with
deliberate practice over traditional educational
strategies
•
Design a mastery learning program for a sample
How will we get there?
•
Use clinical skills data to highlight problems with
traditional training
•
Discuss how mastery learning is better than
traditional training
•
Review steps to develop a mastery learning
Most Education Interventions
Pretest
Training
Posttest
Typical Educational Outcomes
Mastery Learning: A Better Approach
Pretest
Lecture, video, demonstration and deliberate practicePosttest
FAIL PASSMastery Learning
Clinical Outcomes
Mastery Learning Outcomes
Barsuk et al. Critical Care Medicine 2009 p<.005 50.6 (23.4) (26.8) 48.3 95.9 (5.1) 97.4 (3.5) 0 10 20 30 40 50 60 70 80 90 100
Internal Jugular Subclavian
% C o rre c t Pre Post MPS
Clinical Outcomes: Infections
Cohen et al. Simulation in Healthcare 2010
T3 OUTCOMES: BENEFITS TO SOCIETY
•
The total annual estimated savings
were approximately $820,000, 139
patient hospital days, and 120 MICU
days
•
When compared with the cost of
our intervention ($112,000), the net
savings was approximately
$708,000 (a 7:1 rate of return)
How will we get there?
•
Use clinical skills data to highlight problems with
traditional training
•
Discuss how mastery learning is better than
traditional training
•
Review steps to develop a mastery learning
curriculum
a.
Deliberate Practice
b.
Rigorous Assessment
Goal: Educate Superb Clinicians
• Effective & Safe
• Quality Patient Care
Key Medical Education Research
Concepts
• Mastery Learning
• Deliberate Practice
Mastery Learning
Features
1. Baseline, i.e., diagnostic testing;
2. Clear learning objectives, units ordered by difficulty;
3. Educational activities (e.g., deliberate skills practice) focused on objectives; 4. Minimum passing mastery standard (MPS) for each unit;
5. Formative testing → mastery of each unit; 6. Advancement if performance ≥ MPS; or
7. Continued practice or study until MPS is reached 8. Time varies, outcomes are uniform
Deliberate Practice (DP)
Features
1. Highly motivated learners with good concentration;
2. Engagement with a well-defined learning objective or task; at an 3. Appropriate level of difficulty; with
4. Focused, repetitive practice; that leads to
5. Rigorous, precise measurements; that yield
6. Informative feedback from educational sources (e.g., simulators, teachers); and where
7. Trainees also monitor their learning experiences and correct strategies, errors, and levels of understanding, engage in more DP; and continue with
8. Evaluation to reach a mastery standard; and then
9. Advance to another task or unit 10. Goal: constant improvement
Checklist Development
1. Define task
2. Review literature
3. Draft checklist
4. Review by experts
5. Pilot test with feedback
6. Revise checklist
7. Calibrate checklist
8. Revise checklist—periodically review and revise
Setting an MPS
Norcini Med Educ 2003, Downing et al.; Teach Learn Med. 2006
•
Angoff
•
Define a group of
“
borderline
”
trainees who have a 50%
chance of passing
•
Each judge estimates the proportion of borderline trainees
who would correctly perform each item
•
Hofstee
Each panelist records judgments about:
•
Minimum acceptable passing score
•
Maximum acceptable passing score
•
Minimum acceptable failure rate
•
Maximum acceptable failure rate
Rigorous Outcome Measurement
Norcini, Med Educ 2003
Needs
Uses
Reliable Data
Feedback to
Learners
Valid Decisions or
Six Lessons Learned
1.
ML with DP is a key feature of SBME programs aimed at
boosting skill and knowledge acquisition among medical
learners at all levels—
strong, integrated, sustained
2.
Robust measures yielding
reliable data
that permit
valid
decisions or inferences
are a key feature of SBME
3.
Rater training and constant calibration are
essential
4.
Self-assessments are biased, poor relation to performance
measured objectively
5.
ML with DP cannot be done
“
on the cheap
”
6.
Follow-up testing and remediation are needed to
assure
retention
Design your own Mastery Learning
curriculum
•
Learning objectives
•
Teaching method (i.e. simulators)
•
Skills checklist
•
Rater training
•
Lecture, videos, demonstrations
•
Deliberate Practice
•
Standard Setting
Potential topics
• IV insertion
• Foley catheter insertion
• Phlebotomy
• Incision and drainage of wound
• Suturing
• Code blue team response
• Patient handoffs
• Code status discussion
• Patient presentation skills
• Come up with your own!
Putting it all together: Intern boot
camp
•
Interns report 3 days early for orientation
•
Participate in teaching and assessment
•
Must meet minimum standard to start training
-
Cardiac auscultation
-
Paracentesis
-
Lumbar puncture
-
ICU clinical skills
-
Code status discussion
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Cardiac Auscultation Paracentesis Lumbar Puncture ICU Clinical Skills Code Status Discussions Clin ic al S kills E xa m in at io n Sc or es
Historical Controls Boot Camp-Trained Interns MPS†
References
Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB: Simulation-based mastery learning reduces complications during central venous catheter insertion
in a medical intensive care unit. Crit Care Med. 2009;37:2697-2701
Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB: Use of
simulation-based education to reduce catheter-related bloodstream infections.
Arch Intern Med. 2009;169:1420-1423
Cohen ER, Feinglass J, Barsuk JH, et al: Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a
medical intensive care unit. Simul Healthc. 2010;5:98-102
Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med. 2006;18:50-57
Stufflebeam D: Guidelines for developing evaluations checklists: the checklist development checklist (CDC). Available at:
