THE RELATIONSHIP OF MEDICAL STUDENTS’ PRE- MATRICULATE CLINICAL EXPERIENCE TO STEP I MEDICAL BOARD EXAMINATION SCORES
By
BONNIE BOST LASTER Bachelor of Science University of Kansas
Lawrence, Kansas 1995
Submitted to the Faculty of the Graduate College of the Oklahoma State University in partial fulfillment
of the requirements for the Degree of MASTER OF SCIENCE
THE RELATIONSHIP OF MEDICAL STUDENTS’ PRE-MATRICULATE CLINICAL EXPERIENCE TO STEP I MEDICAL BOARD EXAMINATION SCORES
Thesis Approved: Laura L.B. Barnes, Ph.D. Thesis Advisor Dale Fuqua, Ph.D. Katye Perry, Ph.D. A. Gordon Emslie, Ph.D. Dean of the Graduate College
ACKNOWLEDGEMENTS
I would like to express my sincere gratitude to my academic advisor, Laura Barnes. Dr. Barnes has contributed significantly to my success as a student at OSU. Many thanks for your patience, understanding, and long hours of revision overviews! I would also like to thank Dr. Dale Fuqua for opening the world of educational research and statistics to me. Initially gerontology major, I had no interest in statistics whatsoever. One semester of Dr. Fuqua’s introductory statistics and I was captivated! My experience within educational research would not be the same without Dr. Fuqua’s guidance and motivation. Additionally, I would like to thank Dr. Katye Perry for supporting me throughout the last three years. I appreciate the insight Dr. Perry has provided me. Finally, I am grateful my husband, daughter and mother for cheering me on, enduring late nights of revisions, and simply being there when I needed a helping hand. I offer my sincere thanks to each of you.
TABLE OF CONTENTS
Chapter Page
I. INTRODUCTION……….…..1
Background………...….1
Licensing Examinations……….……5
II. REVIEW OF THE LITERATURE……….…….8
The Clinical Skills Correlation……….….….8
Purpose of the Study……….…….11
Significance of Research……….…...13 III. METHODOLOGY………...15 Participants……….…15 Variables…………..………...…15 Method………...….18 IV. RESULTS……….19 General………19 Discussion………...23 V. CONCLUSION Summary……….25 REFERENCES……….….27
APPENDIX A – VOLUNTEER EXPERIENCE……….….31
APPENDIX B – EXPERIENCE REQUIRING A CERTIFICATION OR DEGREE………32
LIST OF TABLES
Table Page
I. Descriptive Statistics for Experience………...20
II. Mean COMLEX scores by Experience and MCAT Levels………21
III. Mean MCAT scores by Experience and MCAT Levels……….……….21
IV. ANOVA Summary Table………22
CHAPTER I
INTRODUCTION
BACKGROUND
Medicine is often viewed as an admirable and compassionate occupation, one responsible for the health and welfare of the community. Multiple applicants flock to medical school application websites each year to start or encourage their journey into medicine. The applicants typically represent large variations in academic preparedness, clinical training and knowledge of the profession (McManus, Richards, Winder & Sproston, 1998). Personal statements from applicants to the Oklahoma State University Center for Health Sciences (OSU-CHS) Doctor of Osteopathic Medicine (D.O.) program range from narratives of those who have held long-term aspirations of medicine,
dreaming of becoming a doctor since the acquisition of a first Fisher-Price doctor’s kit, to those whose inspiration piqued from such trendy television shows as E.R. and Chicago Hope. With 145 medical schools in the United States alone, each producing 77-275 graduates per year, obviously medicine is an attractive vocation for many. However, although medicine is one of the most emotionally and physically demanding professions, few students claim any reasonable knowledge about their chosen career (Marley & Carman, 1999). According to admissions professionals at OSU-CHS, it is the lack of
vocational knowledge, as well as other factors, that encourage medical schools to promote clinical experience from medical school applicants.
Clinical experience may be defined as “authentic human contact in a social or clinical context that enhances learning of health, illness or disease, and the role of the health professional” (Dorman & Bundy, 2004). Most medical schools in the United States strongly recommend, if not require, clinical experience from all applicants. Clinical experience is encouraged from step one, even as prospective applicants prepare for their journey to medical school. Both the American Association of Medical Colleges (AAMC) and the American Association of Colleges of Osteopathic Medicine (AACOM) websites advise applicants to obtain substantial clinical experience prior to application to medical school. The AAMC represents the 125 accredited M.D. granting medical schools in the United States, while AACOM represents the twenty accredited D.O. granting medical schools in the United States (AAMC, 2004; AACOM, 2004). Both websites provide the primary applications for their respective schools. As such, medical school applicants must view these sites at least once as they submit their primary
applications, and are likely to encounter the organizations’ views on clinical experience. Minimally, applicants are encouraged to shadow attending physicians or volunteer in hospitals, clinics and doctor’s offices. Ideally, applicants obtain experience requiring a certification or degree such as a certified nurses assistant, emergency medical technician, registered nurse, and so on. Certified clinical experience typically establishes benefit over volunteering, as volunteer work often consists of such rote tasks as filing or
answering phones. Depending upon the certification obtained, applicants may have the opportunity to draw blood, start IV’s, give injections, and administer life support, or perform other significant clinical procedures (AMA, 2004). Experience obtained by holding a certification also has an obvious benefit of monetary compensation, as most of these positions are occupational in nature. Typically, medical schools encourage clinical experience so admissions staff and faculty can separate applications quantitatively similar, to ensure applicants are familiar with occupation and the demands of it, and to ensure matriculating students have been exposed to working with the public (The Importance of Clinical Experience, 2004).
Recent studies have suggested early clinical experience may have other benefits as well. In her study of undergraduate nursing students, Paskiewicz reported increased motivation in nursing students to continue their professional aspirations to the nursing field. Students informally shadowed nursing faculty in direct patient care settings. Students described the experience as meaningful, as they were able to learn clinically in a non-academic and non-threatening environment. The advantages were three-fold. Students were able to learn and retain clinical information, as well as increase their motivation to the nursing profession, while participating faculty members were able to strengthen their relationships with the students and directly observe their clinical aptitude (2002). A study of the University of Toronto’s medical school found a strong
relationship between level of clinical experience and students’ confidence while in medical school, thereby resulting in a more assured and encouraged student, with less
apprehension of patient contact (Morgan & Cleave- Hogg, 2002). Other studies have found students who actively pursue early clinical exposure were likely to be associated with deep and strategic learning styles. The studies found students who were driven to increase their clinical exposure demonstrated positive, well-organized learning beyond surface memorization (McManus, Richards, Winder & Sproston, 1998; Martin, Stark & Jolly, 2000; Dornan & Bundy, 2004). Well-organized learning styles have an obvious benefit to students, who may be able to absorb and retain information more readily. In their interviews with medical students at the University of Manchester, Dornan and Bundy found clinical experience also eases the synthesis of basic science information by linking it to what the student has observed clinically. This cognitive road map of
previous experience may inspire students to learn and recall biomedical information more readily than rote memorization of pathophysiological facts (2004). Dornan and Bundy’s findings are consistent with some current learning theories. Craik and Lockhart postulate information involving strong visual images or associations with existing knowledge will be processed at a deep cognitive level, thereby making the information more readily retrieved from memory (1972). Bandura’s social learning theory emphasizes learning is largely a social action which takes place observationally through modeling. From observing others, individuals cognitively form a guide for their own actions (Bandura, 1977, as cited in Goldhaber, 2000). The theories from Craik, Lockhart and Bandura give support to the significance of early clinical experience. By observing and modeling medical professionals, prospective medical students can imbed important clinical
concepts even before matriculation into medical school. Once in medical school, these learned concepts may be readily retrieved from memory and applied to the task at hand, whether it is basic academic measurements, or national board examinations. It is the national board examinations to which we now turn.
LICENSING EXAMINATIONS
In the United States, medical students face the daunting task of completing four years of medical school: 2 years of a didactic basic science curriculum, followed by 2 years of hands-on clinical science education. Medical graduates then enter a 3-6 year medical residency to solidify their clinical expertise and learn the specifics of their designated specialty. Throughout the educational process, students and graduates must take medical proficiency licensing exams. Practicing physicians in the United States must be licensed through the National Board of Medical Examiners (NBME) or the National Board of Osteopathic Medical Examiners (NBOME). Both institutions apply rigorous standards to assess a physician's ability to relate scientific and clinical
knowledge, concepts, and principles, and to demonstrate fundamental patient-centered skills, that are central to health, the elimination or control of disease and to safe and effective patient care (NBME, 2004 Licensure is ensured through one of two national board examinations. Comprised of three steps, the United States Medical Licensing Exam (USMLE) is administered by the NBME and required of all practicing M.D.’s. Also comprised of three steps, the Comprehensive Osteopathic Medical Licensing Exam
(COMLEX) is administered by the NBOME and required of all practicing D.O.’s. Comparable in content, results of both tests are reported to the respective medical licensing authorities for their use in granting the initial license to practice medicine. The three comprising steps are administered throughout the four years of medical school and first year of medical residency. Both M.D. and D.O. medical students take Step I the end of the second year of medical school, Step II during the fourth year of medical school and Step III during the first year of residency. In both the USMLE and COMLEX, Step I is of particular importance as it assesses whether the student can comprehend and apply the concepts of sciences basic to the practice of medicine, with special emphasis on
principles and mechanisms underlying health, disease, and modes of therapy. Step I ensures mastery of the foundations for the safe and competent practice of medicine presently, as well as scientific principles required for maintenance of competence through lifelong learning. Because of its foundational significance, all medical schools in the United States hold strict policies regarding the passing of Step I. Although most schools give students the option of repeating the exam if not passed successfully, students are ultimately dismissed if competence is not demonstrated.
It has been found mastery of concepts critical to success on medical board examinations is highly correlated with academic achievement during the medical school curriculum. Many variables contribute to academic achievement, as well as to success on medical board examinations. Thriving students must demonstrate a willingness to learn, well-organized learning styles, and efficient retention techniques. Academic
measurements such as grade point average and MCAT score bear good indications of future success. However, could a non-academic variable, specifically students’ pre-matriculate clinical experience, affect achievement on the USMLE or COMLEX? Research suggests early clinical experience is associated with numerous attributes worthwhile to success in medical school, such as motivation, the ability to relate didactic learning clinically, confidence and profound learning styles. Social learning theory also gives support to clinical experience, stating information is best learned through
observation and modeling. Could these attributes produce the necessary aptitude to yield quantitative success on medical boards? The problem addressed in this study is that as a whole, no current study separates the individual impact of clinical experience on medical board scores. Chapter II introduces a review of the relevant literature with special attention to the association between clinical experience and learning style. Chapter III presents participant demographics and research methodology. Chapter IV presents the study’s findings with special attention to group differences in MCAT and board scores. Chapter V states the study’s conclusion and recommendations for further research.
CHAPTER II
REVIEW OF THE LITERATURE
THE CLINICAL SKILLS CORRELATION
Recent studies have revealed significant effects of prior clinical experience on students’ cognition in multiple ways. In case based learning, clinical experience allows students a significant role in organizing the problem under study and encourages them to consider multiple diagnoses at once. Case based, also known as problem based learning, is a patient based approach to medical education. Students discuss individual patient cases in detail as they relate basic and clinical science knowledge to solve problems through investigation. Students with previous experience may be able to reason with clinical aptitude, as they understand first hand the relevance from a clinical, as well as a basic science point of view. Seeing that case based learning is rapidly becoming the norm in most medical school curricula, this may have a profound effect, as students are able to effectively draw on the knowledge obtained through clinical experience (Sutyak, Lebeau, Spotnitz, O’Donnell & Mehne, 1996). In their interview with first year medical students at the University of Manchester, Dornan and Bundy also found students are readily able to relate experience to the didactic setting. An interviewed student related his own personal experience “It is very much easier to link what you have learnt if you can say: ‘Oh yes, well I saw somebody with that,’ and then that creates a picture in your
mind, and it actually helps you to remember why things are linked that way and what the conditions are, and how they work” (Dornan & Bundy, 2004, pg 8). An interviewed staff member agreed with the importance of experience. In his words, “What I’m quite keen to do is to give students experience of the things that there is evidence of. So, rather than just learning in a lecture that men die younger, they should go out and talk to people about when people die and bring back the evidence” (Dornan & Bundy, 2004, pg 8). The authors also found that information linked to visual images, particularly of patients, was easier for students to understand and recall (2004).
Additional studies have found medical student motivation to learn science is increased through understanding the link to clinical medicine, as basic science
applications appear more relevant and pragmatic (Sweeny, 1999; Yoshioka, Uchida & Kozu, 2003). Outside motivation is often necessary in medical school, as students typically take 24-32 hours of course work per semester throughout the first two years. Clinical rotations throughout the third and fourth years often comprise of long hours and tedious work. The Office of Student Affairs at OSU-CHS reports high student burnout by the end of the second didactic year and again by the middle of the fourth year. Matriculating students who can come in to a medical school curriculum able to relate what they’re learning in basic science to what they’ve experienced clinically may have an advantage due to increased interest and motivation. Kulatunga-Moruzi and Norman found students with early clinical experience may exhibit personal qualities most relevant to the practice of medicine such as integrity, leadership ability, work ethic and orientation
toward service. The relevant qualities are not easily taught and are best obtained through direct experience. Students who pursue clinical opportunities outside the medical school curriculum may reflect a practical pedagogical aptitude, which should be considered a valuable quality in the student selection process (2002). The notion that students obtaining early clinical experience may portray qualities relevant to personal and
professional success is supported by observations from admissions professionals at OSU-CHS (L. Haines, personal communication, 2004). Ms. Haines, the Associate Director of Admissions at OSU-CHS, states applicants who have sought clinical experience prior to matriculation are often hands-on, work-oriented students. These same students often pursue leadership opportunities and community outreach, and are academically at the top of the class. They also note individuals who have gained clinical experience through employment often bring professional maturity to the class, as they have applied experience with professional ethics.
Professional maturity may be an additional variable that influences both experience and academic success. Personal maturity and commitment could affect students’ drive to seek early clinical exposure as they realize early experience could help them both in acceptance into medical school and academically throughout the curriculum. Since AAMC, AACOM and individual medical schools strongly encourage clinical
experience, professional minded students would be apt to take this advice and seek experience accordingly. A professional mindset can also lead to conscientious students, who can appreciate the rigors of medical school and put forward the appropriate amount
of study time. The cognizance of what is needed to be accepted and successful in medical school could be an offshoot of professional maturity, which ultimately leads to an academically thriving student. Perhaps professional maturity also grows from experience with the social, economical and ethnical diversity students commonly
encounter when functioning in a clinical setting. Medicine is a global profession, which encounters individuals from all walks of life. From observing the clinical setting students have the opportunity to appreciate that physicians do not typically select their patients, or their respective ailments. The successful medical professional quickly grasps the
magnitude of working with diverse groups of people (Dornan & Bundy, 2004).
Although the aforementioned studies have established multiple benefits of clinical experience, the specific academic benefit of clinical experience in relation to board scores remains unclear. More precisely, it is unclear whether students with significant clinical experience have an advantage over students without clinical experience in terms of board score outcome.
PURPOSE OF THE STUDY
The purpose of this study is to investigate the relationship between clinical experience and medical board examination scores. Specifically, the study sets out to determine if clinical experience obtained prior to admission to medical school is
associated with osteopathic medical school students’ Step I COMLEX scores. Studies of the correlations between medical school application criteria, academic success and board
scores have been researched at length. Prior research has reported positive correlations between such quantitative variables as the Medical College Admission Test (MCAT), and GPA with academic success and board scores (Brooks, 1981; Jones, Thomae-Forgues, 1984; Basco, Way, Gilbert & Hudson, 2002). Conversely, other research has suggested variables more qualitative in nature do not positively correlate with successful academics and board scores, namely, interview skills, references and personal statements. A cohort study of students at Nottingham Medical School determined the information contained in faculty references did not reliably predict students’ future academic performances
(Ferguson, O'Herir, Sanders & James, 2003). A similar study of medical students throughout the United Kingdom also found references had no predictive value in future academic achievement. Additionally, the authors analyzed the properties of personal statements written by students within their application to medical school. The analysis concluded the material contained in personal statements has little use in predicting academic success (Ferguson, Madeley & James, 2002). However, the majority of research focuses on academic application variables such as MCAT score and
undergraduate GPA, and to a lesser extent, variables non-academic in nature such as communication skills. To an even lesser extent remarkably few studies have been conducted to determine if clinical experience prior to medical school affects high stakes testing of any kind. In their study of outcomes of licensing examinations of the Medical Council of Canada (LMCC), Kulatunga-Moruzi and Norman found traditional cognitive predictors have the most utility in predicting future academic and clinical performances,
in particular GPA and MCAT (2002). Studies assessing the effect of clinical experience on objective structured clinical examinations (OSCE) in the United Kingdom found no clear relationship between clinical experience and OSCE (McManus, Richards, Winder & Sproston, 1998; Martin, Stark & Jolly, 2000). The majority of studies regarding clinical experience and high stakes testing have been conducted outside the United States. Due to the limited research regarding the variables of interest, in particular results of the
COMLEX, this study sets out to determine if pre-matriculate clinical experience is associated with COMLEX board scores, and to what extent.
SIGNIFICANCE OF RESEARCH
The implications of this study affect medical admission professionals, medical school applicants and the community in general. If clinical experience positively affects board scores, medical school admission professionals will have another standard in which to judge applicants. In the 2002-2003 application year, AACOMAS processed 7,134 applications to the 20 colleges of osteopathic medicine, with a 3.37 average GPA and 8.17 average MCAT (www.aacom.org). Quantitatively, applicants are similar. Another admission criterion could ease the burden of selecting a limited number of students on few criteria. The majority of medical schools in the United States hold similar admission standards for potential applicants, namely MCAT, GPA and specific science courses. As such, medical schools already acknowledge these variables affect the quality of
applicants. This current knowledge, in addition to a thorough knowledge of how
previous clinical skills affect student success on national boards could prove invaluable to medical school admission committees. This knowledge could result in a more prepared and competent applicant to medical school. Applicants may strengthen their applications and increase their potential for success on the boards. Ease of basic science assimilation is another potential, as students will be able to relate their clinical experience to the curriculum. Finally, patients can benefit from increased clinical experience, as physicians will be more experienced in clinical procedures.
CHAPTER III
METHODOLOGY
PARTICIPANTS
The study sample consisted of 219 of the 1,986 total graduates of the Doctor of Osteopathic Medicine program at the Oklahoma State University Center for Health Sciences. The sample consisted of all students who graduated 2002, 2003 and 2004. The aforementioned years were selected, as they were the accessible files available through the Office of Student Affairs at OSU-CHS. The average age of the sample was 26, with 46% of the sample female. Ethnicity was 87 % Caucasian, 5 % African American, 3 % Native American, 2.5 % Asian and 2.5 % other. Eighty-seven percent of the students were Oklahoma residents. The average MCAT of the students within the sample was 8.6. The MCAT average nationally is 7.0. Data were collected from admission and academic files of each student and compiled into a separate database.
VARIABLES
Variables utilized for this study were pre-matriculate clinical experience, MCAT scores and Step I COMLEX scores. Each student’s average score on the MCAT was
added as a control, since it could potentially operate as a nuisance variable. Previous studies have found a strong correlation between MCAT scores and medical board scores. Koenig, Sireci and Wiley found that MCAT scores, alone or in combination of
undergraduate GPA were a good predictor of USMLE Step 1 scores (1998). In their correlational study Basco, Way, Gilbert and Hudson found applicant MCAT scores explained 29.1% of the variation in USMLE Step 1 scores (2002).
All applicants to OSU-CHS utilize the AACOMAS application as the initial application to the college. Medical experience was self-reported on each student’s AACOMAS application under volunteer and work sections. Applications of all students in the sample were analyzed to determine if experience had been obtained. Types of experience, or lack of, were quantified into one of three categories: 1. No experience, 2. Volunteer experience and 3. Clinical experience. Experience was considered voluntary if it was listed in the volunteer section of the application and could obviously be performed by a layperson or individual with little clinical training. Examples of volunteer
experience ranged from shadowing physicians to taking patient medical histories at low-income clinics (see Appendix A for a full list of volunteer activities). The length of volunteer time ranged from three months to four years. Experience was considered clinical if it was listed under the employment section and professional in nature. Clinical experience included a range of positions from phlebotomists to physical therapists, pharmacists and veterinarians (see Appendix B for a full list of activities). The length of employment ranged from three months to fourteen years. Within the sample of 219
students, 46 had no experience, 80 had volunteer experience and 93 had experience requiring a certification or degree.
The MCAT scores were reported by the AAMC and recorded on each applicant’s AACOMAS application to OSU-CHS. The average of the three quantitative subscores (biological science, physical science and verbal reasoning) was extracted for each student. The averaged subscores ranged from 6.3 – 12.3 on a scale from
1 – 15 with an overall average of 8.6. For analysis purposes, the student’s average MCAT scores were separated into one of three groups: 1. Low (6.3 – 8.0), 2. Medium (8.3 – 9.0), 3. High (9.3 – 12.3). Scores comprising low, medium and high categories were selected on the basis of national MCAT average (7.0) and OSU-CHS typical MCAT average (8.3 – 9.0). The MCAT was taken by each student at least one year prior to matriculation. Therefore, students could obtain medical experience prior to or after the MCAT was administered.
All students took the COMLEX twice during the four-year curriculum. Step I was administered the end of the second year and Step II was administered during the fourth year. Step I board scores were obtained by each student during their second academic year and reported by the NBOME. The scores were extracted from each student’s permanent academic file for the purposes of this study. The average score for the sample was 532. Nationally, the standard scores of COMLEX Step 1 have a mean of 500 and a standard deviation of 79 in a range of 200-800. A standard score of 400 is required to pass (NBOME, 2004).
METHOD
Medical College Admission Test scores and clinical experience data were
extracted from each student’s AACOMAS application to the school and maintained on a separate database. Step I COMLEX scores were extracted from each student’s
permanent file and maintained on the same database. All names and other identifying information were removed from the data when extracted from the original files, and were randomly inputted into the database; i.e. not in alphabetical order. Measures were taken to maintain confidentiality such that only the primary investigator of the study was initially able to link individual students’ performance with personal identifying information. After the data were randomly entered to the working database, original academic files were no longer utilized. A 3 x 3 factorial analysis of variance (3 levels of clinical experience and 3 levels of MCAT) was conducted via SPSS to determine the effects of level of clinical experience, MCAT average and board scores.
CHAPTER IV
RESULTS
GENERAL
All assumptions of factorial analysis of variance were adequately met. Each student obtained experience, MCAT and COMLEX scores independently. All variances were homogeneous in nature. Histograms of each population found normal distributions for MCAT and COMLEX scores. The distribution for experience was slightly negatively skewed, but compensated by SPSS analysis. For the 3 x 3 analysis of variance, MCAT scores and clinical experience were separated into one of three groups: 1. Low
(6.3 – 8.0), 2. Medium (8.3 – 9.0), 3. High (9.3 – 12.3). Scores comprising low, medium and high categories were selected on the basis of national MCAT average (7.0) and OSU-CHS typical MCAT average (8.3 – 9.0). Clinical experience was quantified into one of three categories: 1. No experience, 2. Volunteer experience and 3. Clinical Experience. As none of the variables were randomized, a factorial analysis of variance with MCAT added for means of control was utilized in lieu of an analysis of covariance.
For the level of no experience, the mean MCAT was 8.6 and mean COMLEX 528. The mean MCAT for volunteer experience was 8.7 with a mean COMLEX of 527. The mean MCAT and COMLEX for experience requiring certification or a degree was 8.6 and 537 respectively.
Results of the analysis of variance indicated the interaction of MCAT scores and level of clinical experience was not statistically significant, F (4, 210) = .519, p =.72 with an effect size of .010. Independently, level of clinical experience was not statistically significant, F (2,210) = .990, p =.37 with effect size .009. As prior research suggested, MCAT average was statistically significant, F (2, 210) = 6.32, p=.002 with effect size .057. The results support previous findings that MCAT strongly affects board scores. Intuitively, this makes sense as both the MCAT and board examinations are high stakes standardized tests. All things equal, students scoring successfully on one should expect to do relatively well on the other. Although few studies of qualitative data exist, the non-significance of obtained clinical experience of this study supports findings that these types of data (personal interview, personal statement, etc.) do not ultimately affect board scores.
Table 1
Descriptive Statistics for Experience
Variable Type N Mean MCAT Std. Deviation Mean COMLEX Std. Deviation No Experience 46 8.6 1.1 528 61.8 Volunteer Experience 80 8.7 1.1 527 67.9 Clinical Experience 93 8.6 1.1 537 61.1 Overall 219 8.6 1.1 532 63.2
Table 2
Mean COMLEX scores by Experience and MCAT Levels
Variable Type Low MCAT SD N Medium MCAT SD N High MCAT SD N No Experience 501.77 69.04 13 545.1 49.27 20 527.23 67.06 13 Volunteer Experience 501.59 61.79 27 530.26 67.02 26 549.67 68.35 27 Clinical Experience 518.97 54.68 34 547.9 52.67 34 547.88 74.97 25 MCAT Levels – Low (6.3-8.0) Medium (8.3-9.0) High (9.3-12.3)
Table 3
Mean MCAT scores by Experience and MCAT Levels
Variable Type Low MCAT SD N Medium MCAT SD N High MCAT SD N No Experience 7.26 .72 13 8.68 .31 20 9.9 .45 13 Volunteer Experience 7.60 .50 27 8.55 .27 26 9.96 .85 27 Clinical Experience 7.48 .57 34 8.69 .29 34 9.96 .77 25 MCAT Levels – Low (6.3-8.0) Medium (8.3-9.0) High (9.3-12.3)
Table 4
ANOVA Summary Table
Dependent Variable: Boards Source
Type III Sum
of Squares df Mean Square F Sig.
Partial Eta Squared
Corrected Model 69728.043(a) 8 8716.005 2.245 .026 .079
Intercept 54808137.14 4 1 54808137.144 14114.230 .000 .985 MCAT 49129.833 2 24564.916 6.326 .002 .057 Experience 7686.776 2 3843.388 .990 .373 .009 MCAT * Experience 8055.192 4 2013.798 .519 .722 .010 Error 815468.395 210 3883.183 Total 62774919.00 0 219 Corrected Total 885196.438 218
a R Squared = .079 (Adjusted R Squared = .044)
DISCUSSION
Pre-matriculate clinical experience did not statistically affect Step I COMLEX scores. As concluded from previous research, the current study found performance on the MCAT is strongly associated with Step I COMLEX scores. In general, the COMLEX scores obtained by low scoring MCAT examinees are lesser than those obtained by medium or high scoring MCAT examinees. Within the low and medium MCAT scoring groups, the certified or degreed experience level had higher board scores. A particularly large difference occurred within the low MCAT scoring group. Students with experience requiring certification or a degree scored on average 17.29 points higher on COMLEX than those with volunteer or no experience. Although the interaction is not significant, it
does generate questions as to the advantage of clinical experience within lower MCAT scoring examinees. Future research could expand upon the relationship between levels of clinical experience within low scoring MCAT groups.
The fact that clinical experience was not significant in regard to board scores is somewhat surprising in one aspect. Seeing that clinical experience is so closely tied to strategic learning styles and motivation, one would expect these characteristics to reflect positively upon board scores. A strategic and motivated learner would be expected to comprehend and maintain the basic science concepts critical to success on Step I, particularly due to the fact that students can readily relate basic science concepts to previously obtained clinical experiences. On the other hand, perhaps students with significant clinical experience are strategic and motivated learners purely in regard to clinical competence. The content of Step I COMLEX never claimed to emphasize clinical ability. According to the NBOME website, in Step I students are expected to demonstrate basic science knowledge relevant to medical problems, emphasizing the medical concepts and principles necessary for understanding the mechanisms of these problems and disease processes. Step 1 covers the basic medical sciences of anatomy, behavioral science, biochemistry, microbiology, osteopathic principles, pathology, pharmacology, physiology and other areas relevant to medical problems. Copied from the NBOME website, Table 4 shows the approximate percentages of physician task content within Step I.
Table 5
Physician Task Percentage on COMLEX
Physician Task (in percentage) Level 1 Level 2 Level 3
Health Promotion/Disease Prevention 1-5 15-20 15-20
History & Physical 5-12 35-40 10-15
Diagnostic Technologies 1-5 15-20 20-25
Management 3-8 10-17 30-35
Scientific Understanding of Mechanisms 75-85 5-8 5-10
Health Care Delivery 0 5-9 5-10
While Steps II and III are designed to more fully assess clinical performance, Step I emphasizes the scientific understanding of mechanisms. Specific clinical tasks such as history taking, physical examination and health care delivery clearly emphasizes less in Step I, than Steps II and III. While pre-matriculate clinical experience certainly has its benefits, it does not show benefit in relation to Step I board scores. Additional studies in regard to pre-matriculate clinical experience and Steps II and III board scores could prove favorable.
CHAPTER V
CONCLUSIONS
SUMMARY
Although clinical experience did not statistically affect Step I board scores, other benefits may be gleaned. Research has determined first patient encounters are often viewed as anxiety provoking and confusing by student doctors (Pitkala & Mantyranta, 2004). Students who matriculate with prior patient contact may benefit over less
experienced students by encountering less stress and/or anxiety due to familiarity with the clinical patient setting. Medicine is one of the most emotionally and physically
demanding professions, yet only 11% of students claim any reasonable knowledge about their chosen career (Marley & Carman, 1999). To motivate and reduce stress among medical students, a thorough knowledge into what they are embarking is paramount. Previous studies have in fact, found decreased stress among medical school students who have already had clinical exposure (Stewart, Lam, Betson, Wong & Wong, 1999).
In conclusion, multiple variables are important when selecting quality medical school students. Both quantitative and qualitative measures such as MCAT and clinical experience can go hand in hand when ultimately producing a competent, confident physician (Morgan & Cleave-Hogg, 2002). While MCAT and GPA may correlate highly with quantitative outcomes such as board scores, prior clinical experience may result in a
more relaxed and motivated student. There are several limitations to this study. Only Step I board scores were analyzed, making an effect of clinical experience on all steps unclear. Only student records at OSU-CHS were utilized, making generalizations outside of the institution invalid. Other factors may play a considerable role in board
performance such as quality of instruction the first two years of medical school, student burn-out, and availability of academic resources. Additionally, self-selection may play a role in the statistical outcome of this study. Potential students who obtain clinical
experience may discover medicine is not an attractive field for them. These applicants may ultimately abandon their aspirations of medicine for a different field. While the attrition of these applicants can benefit admissions professionals by condensing the applicant pool, it does contribute to restriction of range statistically. Finally, longitudinal studies could illustrate the relationship of delayed effects of clinical experience. This study does not address clinical experience influence past the administration of COMLEX Step I. Clinical experience could potentially affect students throughout their third and fourth years of medical school, as well as residency and practice. As studies of clinical exposure are limited, additional research is warranted.
References
Basco, W.T., Jr., Way, D.P., Gilbert, G.E., Hudson, A. (2002). Undergraduate institutional MCAT scores as predictors of USMLE Step I performance. Academic Medicine, 77 (10), S13-S16.
Brooks, C. (1981). Validity of the new MCAT for predicting GPA and NBME Part I examination performance. Journal of Medical Education, 56 (9), 767-769.
Colliver, J.A., Verhulst, S.J. Williams, R.G. (1989). Using a standardized-patient examination to establish the predictive validity of the MCAT. Academic Medicine, 64 (8), 482-486.
Craig, F. & Lockhart, R. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning & Verbal Behavior, 11, 671-684.
Dornan, T., Bundy, C. (2004). What can experience add to early medical education? Consensus survey. British Medical Journal, 329 (7470), 834-838.
Elam, C.L., Johnson, M.M.S. (1992). Prediction of medical students’ academic performances: Does the admission interview help?
Ferguson, E., O'Herir, F., Sanders, A., James, D. (2003). Pilot
study of the roles of personality, references and personal statements in relation to performance over the five years of a medical degree. British Medical Journal, 326 (7386), 429-433.
Goldhaber, D.E. (2000). Bandura’s social cognitive theory. In Theories of Human Development; Integrative Perspectives (1st ed. pp. 87-107). California: Mayfield Publishing Company.
Jones, R. F., Thomae-Forgues, M. (1984). Validity of the MCAT in predicting performance in the first two years of medical school. Journal of Medical Education, 59 (6), 455-464.
Koenig, J.A., Sireci, S.G., Wiley, A. (1998). Evaluating the predictive validity of MCAT scores across diverse applicant groups. Academic Medicine, 73 (10), 1095-106.
Kulatunga-Moruzi, C., Norman, G.R. (2002). Validity of admissions measures in predicting performance outcomes: The contribution of cognitive and noncognitive dimensions. Teaching and Learning in Medicine, 14 (1), 34-42.
Marley, John, Carman, Ian (1999). Selecting medical students: A case report of the need for change. Medical Education, 33 (6), 455-459
Martin, I. G., Stark, P., Jolly, B. (2000). Benefiting from clinical experience: the influence of learning style and clinical experience on performance in an undergraduate objective structured clinical examination. Medical Education, 34 (7), 530-535.
McManus, I.C., Richards, P., Winder, B.C., Sprotson, K.A. (1998). Clinical experience, performance in final examinations, and learning style in medical students: prospective study. British Medical Journal, 316 (7128), 345-350.
Morgan, P.J., Cleave-Hogg, D. (2002). Comparison between medical students' experience, confidence and competence. Medical Education, 36 (6), 534-540.
Paskiewicz, L. S. (2002). The shadowing experience: Valuing the link between faculty practice and student learning. Journal of Professional Learning, 18 (4), 238-242.
Pitkala, K.H., Mantyranta, T. (2000). Feelings related to first patient experiences in medical school. A qualitative study on students' personal portfolios. Patient Education and Counseling, 54 (2), 171-177.
Stewart, S.M., Lam, T.H., Betson, C.L., Wong, C.M., Wong, AM.P. (1999). A prospective analysis of stress and academic performance in the first two years of medical school. Medical Education, 33 (4), 243-250.
Sutyak, J. P, Labeau, R. B., Spotnitz, A. J., O'Donnell, A. M., Mehne, Paul R. (1996). Role of case structure and prior experience in a case-based surgical clerkship. American Journal of Surgery, 172 (3), 286-290.
Sweeney, G. (1999). The challenge for basic science education in problem based medical curricula. Clinical and Investigative Medicine, 22 (1), 15-23.
Yoshioka, Toshimasa, Uchida, Yoko, Tadahiko, Kozu. (2003). Format of cases affects learning outcomes in first year medical students. Education for Health, 16 (1 ), 59-67.
The importance of clinical experience; Applying to medical school. (2004). Retrieved November 13, 2004, from
APPENDIX A
VOLUNTEER EXPERIENCE Shadowing physicians
Taking medical histories at clinics
Teaching health education to peers on college campuses Nursing home/assisted living – volunteer activity aide
APPENDIX B CLINICAL EXPERIENCE
Paramedic/EMT Veterinarian
Registered Nurse Veterinarian Tech
Dietitian Physician’s Assistant
Physical Therapist
Physical Therapy Assistant Pharmacist
Pharmacy Tech Surgical Tech ER Tech
Medical Assistant
Certified Nurse’s Assistant Patient Care Technician Phlebotomist
Athletic Trainer Health Educator Radiology Tech
Oklahoma State University Institutional Review Board
Date: Monday, October 18, 2004
IRB Application No ED0531
Proposal Title:
The Effect of Pre-matriculate Clinical Experience on Basic Science Grades, Step 1 and Step 2 NBOME Scores
Reviewed and Processed as: Exempt
Status Recommended by Reviewer(s): Approved Protocol Expires: 10/17/2005
Principal Investigator(s Bonnie Laster
608 South Redbud Broken Arrow, OK 74012 Laura Barnes
700 N. Greenwood Main H Tulsa, OK 74145
The IRB application referenced above has been approved. It is the judgment of the reviewers that the rights and welfare of individuals who may be asked to participate in this study will be respected, and that the research will be conducted in a manner consistent with the IRB requirements as outlined in section 45 CFR 46.
The final versions of any printed recruitment, consent and assent documents bearing the IRB approval stamp are attached to this letter. These are the versions that must be used during the study.
As Principal Investigator, it is your responsibility to do the following:
1. Conduct this study exactly as it has been approved. Any modifications to the research protocol must be submitted with the appropriate signatures for IRB approval.
2. Submit a request for continuation if the study extends beyond the approval period of one calendar year. This continuation must receive IRB review and approval before the research can continue. 3. Report any adverse events to the IRB Chair promptly. Adverse events are those which are
unanticipated and impact the subjects during the course of this research; and 4. Notify the IRB office in writing when your research project is complete.
Please note that approved protocols are subject to monitoring by the IRB and that the IRB office has the authority to inspect research records associated with this protocol at any time. If you have questions about the IRB procedures or need any assistance from the Board, please contact me in 415 Whitehurst (phone: 405-744-1676, colson@okstate.edu).
Sincerely.
Carol Olson, Chair Institutional Review Board
Bonnie Bost Laster
608 S. Redwood Ave. Broken Arrow, OK 74012 918-231-8622 clint.laster@cox.net EDUCATIONBachelor of Science in Community Health, University of Kansas; May 1995
Completed the Requirements for the Educational Research and Evaluation degree at Oklahoma State University in December 2004
WORK HISTORY
Associate Director of Enrollment and Student Development
Oklahoma State University Center for Health Sciences: Tulsa, OK; March 2004 – present
Responsible for student development in all graduate programs; chairman of student development and student program committees; conduct presentations to pre-health profession college organizations, advisors and faculty; direct all on-campus recruitment and student life events, as well as student graduation; coordinate and conduct campus tours; develop new promotional materials; advise prospective and incoming students; supervise one staff member and four work study students; Assist Director and Associate Dean of Student Affairs with the establishment and evaluation of department goals.
Associate Director of Admission and Recruitment Oklahoma State University Center for Health Sciences: Tulsa, OK; August 1999 – March 2004
Responsible for immediate and
long-range recruitment programs for the Doctor of Osteopathic Medicine program; conducted presentations to premedical college organizations, premedical advisors, health
professions and community organizations; member of the admissions committee for the selection and admission of candidates; directed on-campus recruitment and marketing events, as well as seminars for incoming students; advised prospective and incoming students; coordinated and conducted campus tours; served as staff advisor to the student-driven Osteopathic Run committee; developed new promotional materials; supervised two staff members and three work study students; assisted Associate Dean for Student Affairs with establishment and evaluation of department goals.
Diabetes Program Coordinator
Tulsa Regional Medical Center: Tulsa, OK; January 1998 – April 1999
Responsible for all aspects of The Center for Diabetes Education at Tulsa Regional Medical Center, including management, curriculum and protocol; educated inpatients, outpatients, the community and staff
comprehensive diabetes management; educated patients at the Oklahoma State University Physicians Clinic;
developed and implemented education materials; directed physician advisory board; responsible for marketing and public relations of the program; responsible for
re-certification of the program by the American Diabetes Association; coordinated clinical support for One Touch Hospital Glucometers; performed blood pressure and glucose checks; assisted with Quality Control and Joint Commission committees.
WORK HISTORY CONTINUED
Regional Director
American Heart Association, Tulsa Division: Tulsa, OK; May 1995 – December 1997
Directed AHA Worksite, Schoolsite, Communitysite and Healthcaresite programs for Tulsa County; responsible for a $270,000 campaign; recruited, coordinated and
supervised volunteer, fundraising and public relation programs; worked extensively with school, community, and employee wellness programs; educated public regarding various cardiovascular subjects and performed blood pressure checks; reached approximately 60,000 Tulsa residents with community education.
Activity Aid
Adult Day Program, Douglas County Senior Services: Lawrence, KS; August 1994 – May 1995
Assisted with the planning and implementation of programs for senior citizens, including exercise, motor and cognitive skill activities and rehabilitation; participated in
identification of objectives, goal setting and the evaluation of the program.
Intern
American Heart Association, Kansas City Metro Division: Overland Park, KS;
January 1995 – May 1995
Developed educational materials; assisted with fundraising and community health events; attended Grand Rounds seminars; presented community education programs regarding heart health, including aspects of diet, exercise and stress management; shadowed physicians, nurses, physical therapists and dietitians.
Health Educator
Watkins Health Center, The University of Kansas: Lawrence, KS; April 1994 – May 1995
Educated all aspects of AIDS and sexually transmitted diseases to community health and psychology classes at the University of Kansas, including pathophysiology, transmission, symptoms, and preventative practices. Exercise Physiology Intern
St. John Hospital: Tulsa, OK; May 1993 – August 1993 One of three interns selected; Performed cardiovascular function tests, including treadmill testing, CO2 Max tests, blood pressure and pulse checks; performed
anthropometric measurement tests, advised exercise participants on proper stretching and exercise techniques; developed exercise prescriptions; recorded data; assisted with smoking cessation, weight loss and cooking
instruction; shadowed exercise physiologists, physical therapists and dietitians.
HONORS/MEMBERSHIPS
OSU Leadership Development; 2004 Phi Kappa Phi; 2003 - present
Golden Key National Honor Society; 1993 - present Council of Osteopathic Medicine Admissions Officers; 2002-present
National Association of Graduate Admissions Professionals; 2000 – present
Southern Association of Collegiate Registrars and Admissions Officers; 2000 – present
Kappa Kappa Gamma Alumni Association; 1995 – present
PRESENTATIONS
American Medical Student Association Fall Conference, 2002; “The Difference a D.O. Makes” OACRAO Fall Conference, 2002; “Utilizing Student Ambassadors for Effective College Recruitment” SNMA Medical Conference, 2002; “The D.O. Perspective”
NAGAP Annual Conference, 2003 and 2004; “Utilizing Student Ambassadors for Effective College Recruitment”
TEACHING ASSISTANTSHIPS
Oklahoma State University, 2003-2004; HONR 1000 “Medicine in the 21st Century”
Oklahoma State University Center for Health Sciences, 2004; BIOM 6010 “Research Methods and Design”
CIVIC ACTIVITIES
Child Abuse Network – Fashion Show Volunteer, Tulsa, OK; 2003
Iron Gate Feeding Ministry –Volunteer; Tulsa, OK; May 2002 - 2003
OSU Center for Health Sciences –Osteopathic Run Volunteer; Tulsa, OK; 2000 - 2004
The Nature Conservancy –Logistics Co-chair; Tulsa, OK; 2000 - 2002
The Episcopal Diocese of Oklahoma –Licensed Lay Reader and Sub-Deacon; Tulsa, OK; 1999 – present
The Junior League of Tulsa –Training and Pipeline Committees; Tulsa, OK; 1999 - 2002 Tulsa Regional Medical Center –United Way Campaign Chairman; Tulsa, OK; 1998
Tulsa Regional Medical Center –Chaplain Assistant; Tulsa, OK; 1998 - 1999
Downtown Tulsa Christian Business and Professional Women –President; Tulsa, OK; 1997 - 1998
Tulsa Ballet – Strategic Planning Task Force; Tulsa OK; 1996 - 2001
Downtown Tulsa Christian Business and
Professional Women –Vice President; Tulsa, OK; 1996 - 1997
