Miniprojekt, pedagogisk kurs för universitetslärare I, vt 2006 Inga Soveri, Institutionen för medicinska vetenskaper
Does PBL make better doctors?
Brief literature overview on evidence
Contents
Introduction 3
What is PBL? 3
Advantages and Disadvantages 4
Advantages of PBL 5
Disadvantages of PBL 6
PBL in Undergraduate Medical Education 7
Pre-Clinical Education 7
Clinical Education 10
PBL in Continuing Medical Education 13
Summary 14
Appedix 1 15
Appendix 2 16
References 17
Introduction
As clinical professionals, we rely on guidelines based on evidence, or at least on long term successful experience. When teaching, we often do with our learners what had been done with us. Recognition of the need for change (Högskolverkets utvärdering 1997, Fakultetens självevaluering 1997, AT-läkarenkät 1999,
studentattitydundersökningar, Pedagogisk idéprogram 1999 and
Lärarattitydundersökning 2000) has resulted in implementation of PBL in medical education 1. The need for PBL in medical education is based to the assumption that it teaches the habits of life-long learning. Yet, how much evidence is there that PBL graduates make better physicians?
What is PBL?
”My method is to lead my students by hand to the practice of medicine, taking them every day to see patients in the public hospital, that they may hear the patients´ symptoms and see their physical findings. Then I question the students as to what they have noted in the patient and about their thoughts and perceptions regarding the cause of the illness and the principles of treatment”
Dr Franciscus de la Boe Sylvius, 17th century professor at the University of Leyden, Netherlands 2.
The use of stories or cases to teach has been around for thousands of years. PBL is a version of case-based learning. However, PBL is not about problem solving per se, but rather it uses appropriate problems to increase knowledge and understanding. In PBL, students use triggers from the problem case or scenario to define their own learning objectives. That is a very desirable feature and well in accordance with Knowles´ principles of adult learning (Appendix 1) 2. Subsequently, the students do independent, self directed study before returning to the group to discuss and refine their acquired knowledge.
Advantages and Disadvantages
In 1984, Benjamin Bloom argued that the optimal teaching method is one-to-one tutoring, which would result in the maximum effect, and that all other teaching approaches – such as PBL – are simply attempts to approximate the ideal 3. Table 1 presents differences in teaching methods.
Table 1. Differences in teaching 2
Lecture Seminar PBL Clinical tutorial
One to one clinical
attachment
Efficiency1 ++++ +++ ++ ++ +
Active learning ++ variable ++++ +++-++++ +++++ Mutual feedback ++ +++ ++++ +++-++++ +++++ Modelling behaviour
in real life setting
++ ++ +++ ++++ +++++
+++++ = very high + = very low
1
Advantages of PBL
Group learning facilitates not only the acquisition of knowledge but also
communication skills, teamwork, problem solving, independent responsibility for learning, shearing information, respect for others and other generic skills and attitudes (Appendix 2) 2. Presentation of clinical material serves as stimulus for learning and enables students to understand the relevance of underlying scientific knowledge and principles of clinical practice. The advantages of PBL are listed below:
• Student centered – it fosters active learning, improved understanding, and retention and development of lifelong learning skills
• Generic components – PBL allows students to develop generic skills and attitudes desirable in their future practice
• Integration – PBL facilitates an integrated core curriculum including basic science material into clinical education.
• Motivation – PBL is fun for students and tutors, and the process requires all students to be engaged in the learning process
• “Deep” learning – students interact with learning materials, relate concepts to everyday activities, and improve their understanding
• Constructivist approach – students activate prior knowledge and build on existing conceptual knowledge frameworks
Disadvantages of PBL
However, some evidence suggests that PBL reduces acquisition of basic science knowledge 4. It has also been discussed, that the problems encountered in PBL are due to poor implementation of PBL 5. For instance, problems that should stimulate
learning are too close-ended and simple or they are not realistic. Too dominant and too passive tutors both hinder the learning process as do dysfunctional tutorial groups 5. Disadvantages of PBL are listed in short below.
• Tutors who enjoy passing on their own knowledge and compromise PBL • Human resources – More staff have to take part in the tutoring process.
Setting up PBL courses can be expensive, demanding and stressful 6. • Other resources – Large numbers of students need access to the same library
and computers simultaneously
• Role models – Students may be deprived access to a particular inspirational teacher who in a traditional curriculum would deliver lectures to a large group
• Information overload – Students may be unsure how much self directed study to do and what information is relevant and useful
• Traditional knowledge based assessments of curriculum outcomes have shown little or no differences in students graduating from PBL or traditional curriculums
PBL is supposed to enhance the integration of students´ knowledge. Students use clinical problems as a starting point of the learning process and define their own learning objectives in tutorial groups. These learning objectives reflect basic science disciplines as well as clinical disciplines and both fields are studied concurrently. PBL allows for the horizontal as well as vertical integration of different disciplines. Because learning takes place in a meaningful and authentic context, using clinical cases, for example, students learn to connect clinical phenomena to underlying basic science concepts. Evidence from cognitive psychology has shown that integration of knowledge facilitates the storage and lateral retrieval of relevant information. Integrated knowledge should prepare students better for actual clinical practice 7. Pre-Clinical Education
Medical educators find PBL a more challenging, motivating and enjoyable way to learn, and students appear to agree. Vernon et al studied student satisfaction and supported the superiority of PBL over more traditional methods 4. The educational superiority of PBL relative to traditional approaches is less clear. Given the somewhat extensive resources required for the operation of PBL curriculum, this gives reason for concern. Berkson et al concluded that PBL graduates are not distinguishable from his or her traditional counterpart and the implementation of PBL may be unrealistically costly 6.
Hemker et al observed an uncomfortable role of basic scientists in the PBL curriculum 8
. While organizing the curricula around clinical problems may make the material more relevant, this comes with a cost. Hemker suspects that the knowledge acquired by the students remains badly organized. Some, but not all, disciplines build in a pyramidal form from basic science concepts to more advanced concepts. When one organizes a curriculum around problems, it is very difficult to ensure that concepts will be encountered in the appropriate sequence in those areas where it really matters. As a consequence, students may tend to study at a surface level, looking for definitions rather than concepts. Albanese et al recommended that caution should be exercised in making comprehensive, curriculum-wide conversions to PBL 9.
A number of studies have compared basic science knowledge of PBL students and non-PBL students. When differences in basic science knowledge are found, these are usually in favor of non-PBL students 4, 9-11. Colliver et al reviewed publications on the effectiveness of PBL and found no convincing evidence that PBL improves knowledge base or clinical performance, at least not of the magnitude that would be expected given the resources required for the PBL curriculum 12. Newman et al reviewed randomized controlled trials on the effect of PBL and concluded that outcomes for students in the PBL groups were less favorable than those in control group 13. However, Michel et al found that switch from lecture based learning to PBL teaching in pharmacology does not occur at the expense of factual knowledge and, if anything, the students do slightly better 14. Prince et al found that at start of clinical clerkship, PBL students have same level of anatomy knowledge as students from schools with traditional educational approaches 15. Dochy et al demonstrated that there is positive effect from PBL on skills (i. e. knowledge application) of students, but no effect on knowledge 16. A study in Harvard Medical School showed no difference in diagnostic reasoning and clinical problem-solving prior to students´ medical clerkship in the students on different curriculums 17. In that study, interpersonal skills were assessed with patient interviews and the comparison tended to favor the PBL students. However, that was a comparison of better PBL students with the lowest-performing standard students. It is hard to interpret studies because they are often not randomized: evidence shows that students who select PBL are generally better students. Therefore the positive effect of many studies is not attributable to the curriculum but to self selection 12.
Richards et al compared students who self selected PBL with those in traditional curriculum 18. At the start of clinical clerkship, the factual knowledge, history taking and physical examination skills as well as ability to derive differential diagnosis and ability to organize and express information favored PBL students. However, the differences were small and can be attributed to the fact that better performing students select PBL curriculum as well as to the fact that the PBL track emphasized the frequent contact with real or simulated patients whereas the traditional curriculum did not. The effect may be due to what the students were doing in the sense of their activities and experiences so that traditional students would quickly catch up as they get more clinical practice. Or the effect is due to how the students were learning in the sense of educational principles and underlying mechanisms that would make their learning better and deeper and stick with them longer 12. Another study in a US medical school showed the superiority of PBL students after seven months of training in providing more accurate diagnoses and generating longer reasoning 19. But, again, that was what the PBL students had been doing during the first months of medical school. The PBL students were also more likely to use more hypothesis-driven reasoning than data driven reasoning; the latter is more characteristic of experts, but hypothesis driven reasoning is what is taught in PBL. The PBL students, then, were simply doing what they had been doing with similar problems, which for most part the traditional students had yet to encounter 12. This gives no evidence that PBL students have a lasting advantage.
PBL began with the hope that students would spend more time learning concepts and less time memorizing facts, and every study on student attitudes suggests that PBL succeeds. By contrast, studies of student performance consistently show no advantage for PBL students, either for knowledge or problem-solving. So all that context learning seems to lead nowhere. The traditional discipline-based curriculum imparts coherent bodies of knowledge, but lacks integration and often relevance. PBL curricula, while maximizing clinical relevance and providing a relevant context for learning, do so at the cost of discipline coherence or even clear statement of learning objectives.
Clinical Education
PBL is generally introduced in the context of a defined core curriculum and integration of basic and clinical practice. It has implications for staffing and learning resources and demands a different approach to timetabling, workload and assessment. PBL is often used to deliver core material in non-clinical parts of the curriculum. Paper based PBL scenarios form the basis of the core curriculum and ensure that all students are exposed to the same problems. Recently, also modified PBL techniques have been introduced into clinical education, with real patients being used as stimulus for learning. Despite the essential ad hoc nature of learning clinical evidence, a “key cases” approach can enable PBL to be used to deliver the core clinical curriculum. A randomized study compared diagnostic accuracy scores for Dutch medical students on different curricula (PBL and traditional), the years of training, curriculum and the interaction of the two showed significant effect 20. Years of training accounted for 74% of variance in the students´ diagnostic ability, curriculum type accounted for only 1% of variance and their interaction for only another 1%. Colliver has translated the effect of curriculum into the years in training and found that the effect of curriculum (PBL) was roughly equivalent to an effect of only an additional three or four weeks of medical-school training 12.
A Canadian study compared the passing rates of free classes (1995 – traditional curriculum, 1996 and 1997 – PBL curriculum) on the Medical Council of Canada Qualifying Examination Part 1 at graduation and found identical passing rates 21. However, the PBL students did better in psychiatry and community health subtests. Results from another study also suggest that in psychiatry, a change to a PBL curriculum is worthwhile 22. So, the effects of PBL may be different in different subjects.
Randomized studies have shown no effect of PBL, maybe even a negative effect of PBL, on US National Board of Medical Examiners tests 23. However, some argue that multiple-choice examinations are not appropriate for testing the effectiveness of PBL. One claims that PBL imparts better and deeper learning such that knowledge is better organized and structured and more accessible to recall. However, the randomized studies show no effect on clinical reasoning and problem solving either, and the differences in interpersonal skills are also only moderate. Some non-randomized studies have shown some effect, but these effects are attributable to selection and outcome measured directly reflect the activities and experiences in curriculum tracks. Rapid advances in medical science make it imperative for practitioners to keep up to date. PBL is said to teach the practice of clinical medicine by requiring students to teach themselves, in order to firmly establish life-long habits of self-directed learning. Shin et al looked at adherence to current clinical practice guidelines for management of hypertension as a function of time since graduation and found no change over time for graduates of a PBL school but a (non-significant) decreasing trend for graduates of a standard curriculum school, suggesting that PBL graduates were keeping more up to date 24.
Based on educational theory, PBL should have a positive and sizable effect on the acquisition of basic knowledge and clinical skills 25, 26. Yet the applied research on PBL curricula shows little evidence for the practical effectiveness of PBL in fostering the acquisition of basic knowledge and clinical skills 12. Alternatives to PBL need to be considered. For instance, in teaching critical care, full-scale simulation has been found to be superior to interactive PBL 27. In that way, increase in clinical competence can be increased in realistic environment without fear of patient harm.
PBL in Continuing Medical Education
Adult learners are usually motivated to learn by internal drives rather than by external ones, especially when learning integrates with the demands of their everyday life. Use of PBL in continuing medical education is less common than in undergraduate medical education. Berkson suggests that practicing physicians may be more effective using a problem-solving format than novices 6. A review of studies of variable quality found no consistent evidence that PBL in continuing medical education was superior to other educational strategies in increasing doctors´ knowledge and performance, but there was moderate evidence that it led to higher satisfaction. Limited evidence was found that PBL increased doctors´ knowledge and performance and patients´ health more than no educational intervention at all 28. White et al found no evidence that PBL was any better than more didactic learning sessions in knowledge gain, knowledge retention or changes in attitude about asthma management among primary care physicians 29. However, PBL participants rated the perceived educational value of the program higher than did lecture participants.
Summary
If we are asking whether PBL leads to greater participant enjoyment and enthusiasm for learning than more “traditional” methods of medical education, then there is plenty of evidence to support this: PBL learners feel they are being treated as mature
professionals who are developing effective and clinically relevant skills as well as useful skills in problem solving that are vital in their working life. They also value the interpersonal skills that PBL encourages and that are also keys to effective clinical practice. There is practically no evidence, however, that PBL graduates demonstrate improved clinical competence or have more effective clinical consultations, although it might be argued that in becoming more confident and aware as professional learners, the students will presumably become more efficient and enthusiastic in the workplace 30
. In 2000, the concept of Best Evidence Medical Education was published emphasizing the need to bring educational practitioners and researchers together 31.
One cannot help but notice the similarities in the concepts of Best Evidence Medical Education and Evidence Based Medicine. Effects of PBL may be different in different subjects. There is need for research to establish whether the theoretical benefits of PBL also show solid positive effect in practice and produce better physicians. Also, alternatives to PBL as well as traditional educational methods should be actively explored.
Appendix 1
Knowles´ seven principles on how to teach learners who are somewhat independent and self directed (adults) 2
1. Establish effective learning climate, where learners feel safe and comfortable expressing themselves
2. Involve learners in mutual planning of relevant methods and curricular content
3. Involve learners in diagnosing their own needs – this will help to trigger internal motivation
4. Encourage learners to formulate their own learning objectives – this gives them more control of their learning
5. Encourage learners to identify resources and devise strategies for using the resources to achieve their objectives
6. Support learners in carrying out their plans
7. Involve learners in evaluating their own learning – this can develop their skills of critical reflection
Appendix 2
Generic skills and attitudes learnt in PBL 2
Teamwork Chairing a group Listening Recording Cooperation
Respect for colleagues´ views Critical evaluation of literature
Self directed learning and use of resources Presentation skills
References
1. om B. The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educational Leadership 1984; 4:4-17.
4. Vernon DT, Blake RL. Does problem-based learning work? A meta-anWeb-page.
http://www.medfarm.uu.se/grundutbildning/framtidens_lakarutbildning/bakgr undsdok.shtml, Accessed 2006-02-21.
2. Cantillon P, Wood D, Hutchinson L. ABC of Learning and Teaching in Medicine: BMJ books, 2003.
3. Bloalysis of evaluative research. Acad Med 1993; 68:550-63.
5. Dolmans DH, De Grave W, Wolfhagen IH, van der Vleuten CP. Problem-based learning: future challenges for educational practice and research. Med Educ 2005; 39:732-41.
6. Berkson L. Problem-based learning: have the expectations been met? Acad Med 1993; 68:S79-88.
7. Regehr G, Norman GR. Issues in cognitive psychology: implications for professional education. Acad Med 1996; 71:988-1001.
8. Hemker H. Critical Perceptions on Problem-based Learning. Advances in Health Sciences Education 1998; 3:71-76.
9. Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med 1993; 68:52-81. 10. Schmidt HG, Dauphinee WD, Patel VL. Comparing the effects of
problem-based and conventional curricula in an international sample. J Med Educ 1987; 62:305-15.
11. Verhoeven BH, Verwijnen GM, Scherpbier AJJA, et al. An analysis of progress test results of PBL and non-PBL students. Medical Teacher 1998; 20:310-316.
12. Colliver JA. Effectiveness of problem-based learning curricula: research and theory. Acad Med 2000; 75:259-66.
13. Newman M. A pilot systematic review and meta-analysis on the effectiveness of problem-based learning. On behalf of the Campbell Collaboration
Systematic Review Group on the Effectiveness of Problem-based Learning. Newcastle upon Tyne, UK: Learning and Teaching Support Network-01, University of Newcastle upon Tyne. 2003.
14. Michel MC, Bischoff A, Zu Heringdorf M, Neumann D, Jakobs KH. Problem- vs. lecture-based pharmacology teaching in a German medical school. Naunyn Schmiedebergs Arch Pharmacol 2002; 366:64-8.
15. Prince KJ, van Mameren H, Hylkema N, Drukker J, Scherpbier AJ, van der Vleuten CP. Does problem-based learning lead to deficiencies in basic science knowledge? An empirical case on anatomy. Med Educ 2003; 37:15-21.
16. Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: a meta-analysis. Learning and Instruction 2003; 13:533-568.
17. Moore GT, Block SD, Style CB, Mitchell R. The influence of the New Pathway curriculum on Harvard medical students. Acad Med 1994; 69:983-9. 18. Richards BF, Ober KP, Cariaga-Lo L, et al. Ratings of students' performances
in a third-year internal medicine clerkship: a comparison between problem-based and lecture-problem-based curricula. Acad Med 1996; 71:187-9.
19. Hmelo C. Cognitive consequences of problem-based learning for the early development of medical expertise. Teach Learn Med 1998; 10:92-100. 20. Schmidt HG, Machiels-Bongaerts M, Hermans H, ten Cate TJ, Venekamp R,
Boshuizen HP. The development of diagnostic competence: comparison of a problem-based, an integrated, and a conventional medical curriculum. Acad Med 1996; 71:658-64.
21. Kaufman DM, Mann KV. Comparing achievement on the Medical Council of Canada Qualifying Examination Part I of students in conventional and problem-based learning curricula. Acad Med 1998; 73:1211-3.
22. McParland M, Noble LM, Livingston G. The effectiveness of problem-based learning compared to traditional teaching in undergraduate psychiatry. Med Educ 2004; 38:859-67.
23. Mennin SP, Friedman M, Skipper B, Kalishman S, Snyder J. Performances on the NBME I, II, and III by medical students in the problem-based learning and conventional tracks at the University of New Mexico. Acad Med 1993; 68:616-24.
24. Shin JH, Haynes RB, Johnston ME. Effect of problem-based, self-directed undergraduate education on life-long learning. Cmaj 1993; 148:969-76. 25. Norman GR, Schmidt HG. The psychological basis of problem-based
learning: a review of the evidence. Acad Med 1992; 67:557-65. 26. Schmidt HG, Norman GR, Boshuizen HP. A cognitive perspective on
medical expertise: theory and implication. Acad Med 1990; 65:611-21. 27. Steadman RH, Coates WC, Huang YM, et al. Simulation-based training is
superior to problem-based learning for the acquisition of critical assessment and management skills. Crit Care Med 2006; 34:151-7.
28. Smits PB, Verbeek JH, de Buisonje CD. Problem based learning in
continuing medical education: a review of controlled evaluation studies. Bmj 2002; 324:153-6.
29. White M, Michaud G, Pachev G, Lirenman D, Kolenc A, FitzGerald JM. Randomized trial of problem-based versus didactic seminars for
disseminating evidence-based guidelines on asthma management to primary care physicians. J Contin Educ Health Prof 2004; 24:237-43.
30. Albanese M. Problem-based learning: why curricula are likely to show little effect on knowledge and clinical skills. Med Educ 2000; 34:729-38.
31. Harden RM, Grant J, Buckley G, Hart IR. Best Evidence Medical Education. Advances in Health Sciences Education 2000; 5:71-90.
