Original Research
Relationship between medical specialty and emergency department rotation
performance of postgraduate year 1 residents
Pin Liu
a, Chung-Hsien Chaou
a, Chien-Kuang Chen
a, Chip-Jin Ng
a, Chien-Hung Liao
b,
Jih-Chang Chen
a, Yu-Che Chang
a,c,d,*
aDepartment of Emergency Medicine, Chang Gung Memorial Hospital, Linkou, Taiwan bDepartment of Traumatology and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Taiwan
cDepartment of Medical Education, Chang Gung Memorial Hospital, Linkou, Taiwan dSchool of Medicine, College of Medicine, Chang Gung University, Taoyuan, Taiwan
Revised 13 March 2014; accepted 25 March 2014 Available online 23 May 2014
Abstract
Background: Before July 2011, all medical graduates chose their specialties for residency training prior to receiving a 1-month postgraduate year 1 (PGY-1) emergency medicine (EM) training in Taiwan. Therefore, the EM curriculum content may not correlate well with the chosen specialty. Accordingly, the PGY-1 trainees might learn from their EM training differently depending on specialty.
Purpose: This study explored the influence of the specialties of PGY-1 trainees on their performance in the 1-month EM training using workplace assessment tools.
Methods: This retrospective study analyzed the clinical performance of 183 PGY-1 residents who underwent a 1-month EM training program in the emergency department of a teaching hospital. Their performance was assessed using several mini-clinical evaluation exercises (mini-CEXs) and a single monthly global rating. We classified trainees into three groups based on the extent to which the specialty chosen reflected primary care. (Radiology, pathology, nuclear medicine, and anesthesiology reflected little correlation. Obstetrics and gynecology, pediatrics, otolaryn-gology, ophthalmology, physical medicine and rehabilitation, psychiatry, neurosurgery, neurology, orthopedics, and urology reflected some correlation. Internal medicine, general surgery, EM, and family medicine had high correlation.) We analyzed the variation in the assessment outcomes between groups using KruskaleWallis test.
Results: Success in achieving learning outcomes in the emergency department was proportional to the degree of general practice of the specialty chosen. This trend was statistically significant for the mini-CEX domains of medical interviewing (p¼ 0.028), clinical judgment (p ¼ 0.012), physical examination (p¼ 0.001), professionalism (p ¼ 0.023), and clinical skills (p ¼ 0.001). Compared to the mini-CEX, the final monthly global rating showed no significant correlation between degree of learning and the trainee specialty.
Conclusion: The success of PGY-1s in achieving learning outcomes after 1-month EM training was correlated with the degree of primary care of the specialty. The structured workplace assessment tool, mini-CEX, provided a more accurate evaluation compared with the single monthly global rating score.
CopyrightÓ 2014, Taiwan Society of Emergency Medicine. Published by Elsevier Taiwan LLC. All rights reserved.
Keywords: clinical competence; educational assessment; emergency departments; emergency medicine; global rating; medical residency; mini-CEX
* Corresponding author. Department of Emergency Medicine, Chang Gung Memorial Hospital, 5 Fushing Street, Gueishan Shiang, Taoyuan 333, Taiwan. E-mail address:changyuche@gmail.com(Y.-C. Chang).
ScienceDirect
Journal of Acute Medicine 4 (2014) 86e91
www.e-jacme.com
http://dx.doi.org/10.1016/j.jacme.2014.03.002
1. Introduction
The mini-clinical evaluation exercise (mini-CEX), a struc-tured and reliable tool used to assess clinical performance, was first developed by the American Board of Internal Medicine in the 1970s.1 Although initially developed only to assess trainees in internal medicine departments,2the mini-CEX has since been widely implemented in many other specialty training programs.3e10Its validity, reliability, and feasibility in assessing the clinical performance of residents in various specialties had been well established.11e13 The tool works well in the evaluation of many types of examinees: medical students, clerks, interns, residents, or even practicing phys-icians.7e9,14e16In a mini-CEX encounter, the faculty member acts as an evaluator to observe the workplace performance of a trainee. The trainee, either a medical student or a resident, performs a focused history taking, physical examination, or medical counseling with a real patient in the clinical setting. After the encounter, the evaluator completes a structured rat-ing form in multiple domains related to the trainee’s clinical competency. In addition to serving as an evaluation tool, the mini-CEX has an educational component; by providing timely and focused feedback to trainees, it also improves the quality of training.17,18
In Taiwan, since 2003 the 3-month postgraduate year 1 (PGY-1) general medicine training program has consisted of monthly rotations in internal medicine, surgery, and commu-nity medicine departments.19 In 2006, the general medicine training was extended to 6 months. From July 2009 to June 2011, the Taiwan Department of Health required all PGY-1 residents to complete a 1-month training course in the emer-gency department (ED), which was integrated into the former 6-month program.20 These PGY-1 residents spend the remaining 6 months of the first year in their own specialty training courses (e.g., neurology or orthopedics). They had already chosen their specialty of residency training prior to graduating from a medical school, and may receive specialty training before or after the general medicine (including ED) training, based on the 1st-year course arranged by the depart-mental program director of the chosen specialty.
No previous study has yet focused on the influence of specialty background on the performance of PGY-1 trainees in the ED. In an era where medical education of the emergency department was highly emphasized, adult learning theory is a crucial factor to be considered in designing emergency med-icine (EM) training curriculum. We therefore hypothesize that the chosen specialties of PGY-1 trainees with a holistic approach and their professions or daily practice highly asso-ciated with EM learning contents will achieve higher mini-CEX performance score in the 1-month EM training. 2. Methods
2.1. Study design and population
This retrospective study analyzed the assessment data of mini-CEX encounters and monthly global ratings of PGY1
residents in a medical center of northern Taiwan. The study period, study population, and data collection of mini-CEXs are the same as described in our previous study, whose aim was to determine the feasibility, validity, and impact factors of the mini-CEX ratings in the ED setting (and was published in the Journal of Acute Medicine). In addition to monthly global rating scales, we analyzed the same raw data of mini-CEXs. We hereby quoted a part of the description of the study period and population from Chang et al’s19study.
All PGY-1 residents who underwent a 1-month training program in the ED between August 2009 and December 2010 were enrolled in our study. Their performances were assessed with the mini-CEX and a final monthly global rating. During the 1-month rotation in EM, the PGY-1 residents underwent 1 week of trauma training conducted by trauma surgeons and 3 weeks of nontrauma training conducted by emergency physi-cians (EPs). The core contents of the 1-month EM training contained general principles of management of common emergencies and traumatic cases, basic life support and advanced cardiac life support skills, emergency medical ser-vices system, emergency and critical transfer, disaster and mass casualty management, ethics issues and medical malpractice in the ED, substance abuse and intoxication, gy-necologic and obstetric emergency, and emergency medical sociology.
The clinical competency of each PGY-1 resident was assessed with multiple mini-CEX encounters, on average 3e6. The evaluator was either an EP or a trauma surgeon, depending on the clinical situation. Each PGY-1 resident also had a single clinical tutor to guide him or her throughout the 1-month EM course. The tutor gave the trainee a single overall (global) assessment at the end of the EM training, using a traditional 100-point scale.
The ED faculty received multiple lectures and video-based workshops on rater training prior to the formal implementation of mini-CEX as an assessment tool for the PGY-1 EM training. Therefore, most evaluators were assumed to be competent to perform this workplace assessment of clinical competency using the tool provided.
Based on the degree to which the chosen specialties used a holistic approach (i.e., one that takes into consideration the biological, psychological, and social environment of the pa-tients) and the association between the teaching contents provided in the 1-month EM training course for the PGY1 trainees and their daily practice or experience required and documented in their residency training programs, we classified the PGY-1 trainees into three groups. Group A (radiology, pathology, nuclear medicine, and anesthesiology) includes medical specialties with less of the character of a general physician using a holistic approach skill, and there was a higher disparity between EM teaching contents provided and trainees’ specialty training required. Group C (internal medi-cine, general surgery, EM, and family medicine) contains specialties whose practice uses a holistic approach such as that of a general physician and lesser disparity between EM teaching contents provided and trainees’ specialty training required. Group B (obstetrics and gynecology, pediatrics,
otolaryngology, ophthalmology, physical medicine and reha-bilitation, psychiatry, neurosurgery, neurology, orthopedics, and urology) contains the intermediate generalist character of physician specialties and disparity between EM teaching contents provided and trainees’ specialty training required, which is considered between those specialties defined in Groups A and C.
2.2. Study protocol and measures
This study was a retrospective review of mini-CEXs and global assessments completed by ED faculty members from August 2009 to December 2010. The mini-CEX is composed of seven domains (medical interviewing, physical examina-tion, professionalism, procedural skills, clinical judgment, counseling skills, and organization) using a 9-point rating scale (1 indicating unsatisfactory and 9 indicating superior). Each PGY-1 resident was evaluated with a mini-CEX weekly. At the end of the month, the tutor gave the resident a single overall score for knowledge, skill, and attitude, using a 100-point scale. The protocol was approved by the Hospital Ethics Committee on Human Research, which deemed it free from the requirement of obtaining documented informed consent.
2.3. Data analysis
The performance in ED of PGY-1 residents by specialty was evaluated using both the mini-CEXs and the monthly global rating. The ShapiroeWilk test was used to test the null hypothesis of normal distribution. The differences in the assessment results in seven individual domains between the
three groups were analyzed using KruskaleWallis test. Post hoc comparisons were made using Dunn test. We also exam-ined the global rating score between groups. All analyses were performed using SAS statistical software version 9.2 (SAS Institute Inc, Cary, NC, USA). A p value of <0.05 was considered statistically significant.
3. Results 3.1. Overview
Over the 17-month period of EM training (from August 2009 to December 2010), 57 faculty members (42 EPs and 15 trauma surgeons) evaluated 183 PGY-1 residents from 18 different specialties, creating a total of 723 mini-CEX en-counters during the study period (Table 1). Of these, 591 encounters (86.8%) were rated by EM physicians and 132 (13.2%) were rated by trauma surgeons. The mini-CEX domain of clinical competency most often achieved by PGY-1 trainees in ED was medical interviewing (99.3%), clinical judgment (99.1%), and physical examination (98.5%); the least achieved domain was procedural skills (42.0%) (Table 2).
3.2. Influence of specialty on trainees’ performance evaluated by mini-CEX
The mean score in each domain of mini-CEX was higher for those who had chosen a primary care specialty. This trend was statistically significant for the mini-CEX domains of medical interviewing (p ¼ 0.028), clinical judgment (p¼ 0.012), physical examination (p ¼ 0.001), professional-ism (p¼ 0.023), and procedural skills (p ¼ 0.001). The trend was not statistically significant for the domains of counseling skills (p ¼ 0.098) and organization/efficiency (p ¼ 0.065)
(Table 3). In post hoc analysis, Group C and Group A
per-formed with significant differences in medical interviewing, clinical judgment, physical examination, professionalism, and procedural skills. Group B and Group A performed with sig-nificant differences in medical interviewing and physical ex-amination. There were no significant differences in performance between Group B and Group C in post hoc analysis.
Table 1
The distribution of PGY1 residents and relevant mini-CEX encounters in different medical specialty.
Examinee specialty Number of examineesa Number of mini-CEX testsa N¼ 183 (100%) N¼ 723 (100%) Internal medicine 49 (26.8) 209 (28.9) General surgery 29 (15.8) 101 (14.0) Pediatrics 17 (9.3) 72 (10.0) Emergency medicine 14 (7.7) 51 (7.1) Obstetrics and gynecology 11 (6.0) 40 (5.5) Physical medicine and rehabilitation 8 (4.4) 31 (4.3)
Orthopedics 7 (3.8) 28 (3.9) Anesthesiology 7 (3.8) 28 (3.9) Neurology 6 (3.3) 24 (3.3) Radiology 5 (2.7) 23 (3.2) Otorhinolaryngology 5 (2.7) 19 (2.6) Neurosurgery 5 (2.7) 17 (2.4) Ophthalmology 4 (2.2) 17 (2.4) Pathology 4 (2.2) 17 (2.4) Family medicine 4 (2.2) 16 (2.2) Psychiatry 4 (2.2) 14 (1.9) Urology 3 (1.6) 12 (1.7) Nuclear medicine 1 (0.5) 4 (0.6)
Mini-CEX¼ mini-clinical evaluation exercise; PGY1 ¼ postgraduate year 1.
a
Data are presented as number (%).
Table 2
Overall mean ratings of each domain of mini-CEX. Domains of mini-CEX Number of each
domain assesseda Overall mean ratingb Medical interviewing 717 (99.2) 6.8 (1.0) Physical examination 712 (98.5) 6.7 (1.0) Clinical skills 313 (43.3) 6.7 (1.1) Counseling skills 645 (89.2) 6.6 (1.1) Clinical judgment 716 (99.0) 6.8 (1.1) Organization/efficiency 696 (96.3) 6.7 (1.1) Professionalism 684 (94.6) 6.9 (1.1) Mini-CEX¼ mini-clinical evaluation exercise; PGY1 ¼ postgraduate year 1.
a
Data presented as number (%).
b
3.3. Influence of specialty on trainees’ performance evaluated by a monthly global rating score
The mean monthly global score of all PGY-1 residents was 86.5 5.6. Those in Group A scored 85.2 5.8, those in Group B scored 86.3 5.5, and those in Group C scored 86.8 5.6. The difference between groups was not statisti-cally significant even though scores were correlated with the extent of primary care focus of the chosen specialty.
4. Discussion
4.1. Factors influencing the career choicesdadult learning theory
From a national survey in the United Kingdom, early career choices for EM are less predictive of career destinations, and those who switched to EM were, notably, doctors who previ-ously favored surgical specialties, hospital physician-led spe-cialties, and anesthetics.21 According to Lefevre et al,22 the main motivating factors for medical students’ choice of career included interesting diseases, opportunities for private prac-tice, patient contact, and quality of life. In the study by Kumar and Dhaliwal,23personal interest was rated by 80% of senior medical students as important in influencing their career choice, followed by stability (58%), reputation of the specialty (56%), and lifestyle (55%). Although a strong interest in certain highly specialized fields of medicine may influence one’s career choice, many medical students cite job avail-ability for the future as their major concern.24As such, adult learning theory noted by Lindeman may be considered highly relevant in interpreting our assessment outcomes.25Physicians
are motivated to learn as they experience needs and interests that learning will satisfy.26These adults have a deep need to be self-directed. It would be logical to assume that physicians with a background in EM, internal medicine, general surgery, or family medicine would be more motivated to learn EM and thus would score higher in tested domains.
Lambert and Goldacre27found that the percentage of PGY-1 doctors whose first choice of eventual career was general practice has not changed much in England in recent years. The career of choice is presumably related to a medical graduate’s personality and individuality, and it may not easily be changed. In theory, those who choose general practice as a future career will presumably perform better in skills related to patient contact (e.g., history taking and physical examination) than those who choose a specialty with fewer doctorepatient interactions. As such, the specialties in Group C require the mandated skills in general medicine training as set by the Taiwan Department of Health for all PGY-1 residents. This fact is compatible with our observation that PGY-1 residents in Group C got higher scores in most domains of mini-CEXs as compared to other groups. The variation in performance of the PGY-1 residents in different domains may reflect their indi-vidual personality, strengths, and weakness. Therefore, the tutors can enhance the training in certain domains according to the background specialty of the trainee.
4.2. Comparison of rating scales in different mini-CEX domains between specialties of the PGY-1 residents
Our study showed that, in the ED setting, PGY-1 trainees with backgrounds in internal medicine, general surgery, EM, or family medicine perform better than those in other spe-cialties in most domains of mini-CEXs, such as medical interviewing, physical examination, procedural skills, clinical judgment, and professionalism in the 1-month EM learning. For trainees with specialties related to primary care (Group C), in the clinical setting, these competencies are closely linked to the daily practice of their profession. Although the trend was similar, the difference between groups was not statistically significant in counseling skills and organization/efficiency. A possible explanation for this homogeneity is the unique teaching context and module in the ED. Patient safety is stressed during the teaching and learning process, and all PGY-1 trainees are instructed in advance and are not allowed to make decisions of disposition during ED visits. Even though all PGY-1 trainees were assigned adequate patient encounters with different acuity and severity levels to achieve the learning goal of EM training, they were supervised by on-site clinical teachers at all times. As most of the trainees were not familiar with the medical resources utilization, system arrangement, and interprofessional collaboration of the teaching environment of the ED, it is difficult for them to provide timely and efficient health care in the ED. It is also difficult for residents in the ED to perform professionally with good clinical reasoning in explaining the rationale for treat-ment, obtaining patient consent, or conducting counseling regarding management in the 1-month EM training course. Table 3
Assessment results in different specialty group of the PGY1 trainees. Group Aa Group Ba Group Ca p
(N¼ 17) (N¼ 70) (N¼ 96) Domains of mini-CEX Medical interviewing 6.4 (1.1) 6.8 (1.0)$ 6.8 (1.0)$ 0.028* Physical examination 6.3 (1.1) 6.7 (1.0)$ 6.8 (1.1)$ 0.001* Clinical skills 6.1 (1.0) 6.7 (1.0) 6.9 (1.2)$ 0.001* Counseling skills 6.4 (1.1) 6.6 (1.0) 6.7 (1.1) 0.098 Clinical judgment 6.5 (1.0) 6.8 (1.0) 6.9 (1.1)$ 0.012* Organization/efficiency 6.4 (1.1) 6.7 (1.0) 6.7 (1.2) 0.065 Professionalism 6.6 (1.1) 6.9 (1.0) 7.0 (1.1)$ 0.023* Monthly global rating score 85.2 (5.8) 86.3 (5.5) 86.8 (5.6) 0.368 Group A: PGY1 trainees with medical specialties of radiology, pathology, nuclear medicine, anesthesiology. Group B: PGY1 trainees with medical specialties of pediatrics, obstetrics and gynecology, physical medicine and rehabilitation, orthopedics, neurology, otorhinolaryngology, neurosurgery, ophthalmology, psychiatry, urology. Group C: PGY1 trainees with medical specialties of internal medicine, general surgery, emergency medicine, family medicine.
Reported p values from Kruskal Wallis test. Post hoc comparisons were made using Dunn’s test.
*Significant difference among three groups.
$
Significantly different from Group A.
Mini-CEX¼ mini-clinical evaluation exercise; PGY1 ¼ postgraduate year 1.
a
These conditions may explain why the domains of counseling skills and organization or efficiency did not differ significantly between PGY-1 trainees of different specialties, regardless of their individual motivation or previous training.
4.3. Comparison of the mini-CEX ratings and the global rating scores
PGY-1 residents who planned to practice primary care did better than the two other groups according to many domains of the mini-CEX. The global rating assessment showed a nonsignificant difference between groups. The main consid-erations for the global rating scale were the ability of PGY-1 residents to handle a medical emergency, skill in basic trauma management, the doctorepatient interaction, attitude in the workplace, and sense of responsibility. The global assessment of the PGY-1 is done by the rater based on the recalled observation of the trainee in the clinical setting at the end of the 1-month training. As expected, the mini-CEX was a more reliable workplace assessment tool. Its superiority lies not only in its structured dimensions in accordance with the clinical competency of ED residents, but also in its completion by raters at the moment of direct observation.
4.4. Contribution and application
Our study results addressed the fact that the ED rotation performance of PGY-1 residents with specialties whose practice uses a holistic approach such as that of a general physician and lesser disparity between EM teaching contents provided and trainees’ specialty training required will be assessed as superior using mini-CEXs as compared to others, which might be rational in accordance with the concept of adult learning theory. We also found that the structured mini-CEXs outperformed the single global rating score as a tool to evaluate the clinical EM competency of PGY-1 residents. The PGY1 training program has been extended to 1 year after July 2011. All PGY1 residents are required to complete a 1-month ED rotation prior to being accepted for specialty training. Future studies for analyzing the association between the trainees’ ED rotation performance outcome and their future specialty chosen is indicated. 4.5. Limitations
Our study had several limitations. First, the sample size of this single-center study is too small to establish strong evi-dence for a conclusion. Second, not all mini-CEX records were computerized during the period of study (some were written on paper). The ED setting, with its limited time and stressful environment, made it difficult for the ED faculty to fill in these evaluation forms without some error.28Therefore, some of the information may have been left out owing to a lack of computerized, “missing-proof” mechanism.19 Third, although we tried to analyze the influence of background specialty on performance in each domain of PGY-1 residents, performance may have been affected by the length of exposure to the material in each resident’s specialty field prior to the
1-month EM training program. Fourth, the assessment bias that could arise from preconceived preferences on performance of different background specialties should be avoided in the study designment. As this is a retrospective data review, we could not make sure that the faculties (evaluators) were un-aware of trainee specialty. However, the clinical competency of each PGY-1 resident was assessed with multiple mini-CEX encounters, on average 3e6. Therefore, the assessment bias might be limited. Finally, as mentioned by Chang et al,19the seniority and background specialty of ED faculty have some influence on the mini-CEX ratings. PGY-1 residents who were rated by a senior supervisor tended to get lower scores. Other potential influencing factors include the varying complexity of the clinical setting during the mini-CEX encounter17 and the Hawthorne effect.29 The influence of such factors should be taken into account when interpreting the data.
5. Conclusion
Workplace assessment using the mini-CEX indicates that the background specialty training of PGY-1 residents may influence their performance in EM learning. In our study, the structured mini-CEX outperformed the single global rating score as a tool to evaluate the clinical EM competency of PGY-1 residents. Future research on a larger scale, of longer duration, and with more complete data collection, is warranted.
Conflicts of interest
There were no conflicts of interest related to this study. References
1. Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA. 2009;302:1316e1326.
2. American Board of Internal Medicine. Mini-CEX.http://www.abim.org/ program-directors-administrators/assessment-tools/mini-cex.aspx. Accessed 12.02.14.
3. Weller JM, Jolly B, Misur MP, et al. Mini-clinical evaluation exercise in anaesthesia training. Br J Anaesth. 2009;102:633e641.
4. Wiles CM, Dawson K, Hughes TA, et al. Clinical skills evaluation of trainees in a neurology department. Clin Med. 2007;7:365e369. 5. Cohen SN, Farrant PB, Taibjee SM. Assessing the assessments: U.K.
dermatology trainees’ views of the workplace assessment tools. Br J Dermatol. 2009;161:34e39.
6. Cook DA, Beckman TJ, Mandrekar JN, Pankratz VS. Internal structure of mini-CEX scores for internal medicine residents: factor analysis and generalizability. Adv Health Sci Educ Theory Pract. 2010;15:633e645. 7. Wang PW, Cheng CC, Chou FH, et al. Using multiple assessments to
evaluate medical students’ clinical ability in psychiatric clerkships. Acad Psychiatry. 2011;35:307e311.
8. Pernar LI, Peyre SE, Warren LE, et al. Mini-clinical evaluation exercise as a student assessment tool in a surgery clerkship: lessons learned from a 5-year experience. Surgery. 2011;150:272e277.
9. Ferna´ndez Ga´lvez GM. Assessment of clinical competence in a pediatric residency with the Mini-Clinical Evaluation Exercise (Mini-CEX). Arch Argent Pediatr. 2011;109:314e320.
10. Setna Z, Jha V, Boursicot KA, Roberts TE. Evaluating the utility of workplace-based assessment tools for speciality training. Best Pract Res Clin Obstet Gynaecol. 2010;24:767e782.
11. Durning SJ, Cation LJ, Markert RJ, Pangaro LN. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med. 2002;77:900e904.
12. Yang YY, Lee FY, Hsu HC, et al. Validation of the behavior and concept based assessment of professionalism competence in postgraduate first-year residents. J Chin Med Assoc. 2013;76:186e194.
13. Al Ansari A, Ali SK, Donnon T. The construct and criterion validity of the mini-CEX: a meta-analysis of the published research. Acad Med. 2013;88:413e420.
14. Moonen-van Loon JM, Overeem K, Donkers HH, et al. Composite reli-ability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ Theory Pract. 2013;18:1087e1102. 15. Sidhu RS, Hatala R, Barron S, Broudo M, Pachev G, Page G. Reliability
and acceptance of the mini-clinical evaluation exercise as a performance assessment of practicing physicians. Acad Med. 2009;84:S113eS115. 16. Hill F, Kendall K, Galbraith K, Crossley J. Implementing the
under-graduate mini-CEX: a tailored approach at Southampton University. Med Educ. 2009;43:326e334.
17. Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann Intern Med. 2003;138:476e481.
18. Weller JM, Jones A, Merry AF, Jolly B, Saunders D. Investigation of trainee and specialist reactions to the mini-Clinical Evaluation Exercise in anaesthesia: implications for implementation. Br J Anaesth. 2009;103: 524e530.
19. Chang YC, Chen CK, Chen JC, et al. Implementation of the mini-clinical evaluation exercise in postgraduate Year 1 residency training in emer-gency medicine: clinical experience at Chang Gung Memorial Hospital. J Acute Med. 2013.http://dx.doi.org/10.1016/j.jacme.2013.06.004.
20. Lin CS, Chiu TF, Yen DH, Chong CF. Mini-clinical evaluation exercise and feedback on postgraduate trainees in the emergency department: a qualitative content analysis. J Acute Med. 2012;2:1e7.
21. Svirko E, Lambert T, Brand L, Goldacre MJ. Career choices for emer-gency medicine: national surveys of graduates of 1993e2009 from all UK medical schools. Emerg Med J. 2013 Apr 10 [Epub ahead of print]. 22. Lefevre JH, Roupret M, Kerneis S, Karila L. Career choices of medical
students: a national survey of 1780 students. Med Educ. 2010;44: 603e612.
23. Kumar R, Dhaliwal U. Career choices of undergraduate medical students. Natl Med J India. 2011;24:166e169.
24. Leung J, Le H, Turner S, Munro P, Vukolova N. Faculty of Radiation Oncology 2012 trainee survey: perspectives on choice of specialty training and future work practice preferences. J Med Imaging Radiat Oncol. 2013 Oct 10.http://dx.doi.org/10.1111/1754-9485.12105[Epub ahead of print]. 25. Brookfield S. The contribution of Eduard Lindeman to the development of theory and philosophy in adult education. Adult Educ Q. 1984;34: 185e196.
26. Rogers J. Adults learning. Chapter 1. Adult learners: what you need to know. Buckingham: Open University Press; 2007:6e43.
27. Lambert T, Goldacre M. Trends in doctors’ early career choices for general practice in the UK: longitudinal questionnaire surveys. Br J Gen Pract. 2011;61:e397ee403.
28. Yarris LM, Linden JA, Gene Hern H, et al. Attending and resident satisfaction with feedback in the ED. Acad Emerg Med. 2009;16: S76eS81.
29. Holmboe ES, Yepes M, Williams F, Huot SJ. Feedback and the mini clinical evaluation exercise. J Gen Intern Med. 2004;19:558e561.
