Complimentary Contributor Copy

Chapter 11

H

EALTH

P

ROFESSIONS

’

S

TUDENT

A

DMISSION

P

OLICY

:

H

OW TO

L

INK

S

ELECTION AND

P

ERFORMANCE

?

A

S

COPING

R

EVIEW OF THE

L

ITERATURE

Anna Siri

_{, Hicham Khabbache}

_{, Brahim Akdim}

_,

Ali Al-Jafar

_{, Mariano Martini}

and Nicola Luigi Bragazzi

1_{Department of Mathematics (DIMA), University of Genoa, Genoa, Italy} 2_{UNESCO CHAIR “Anthropology of Health –}

Biosphere and Healing System,” University of Genoa, Genoa, Italy

3_{Laboratoire Etudes théologiques, Sciences Cognitives et Sociales,}

Faculty of Literature and Humanistic Studies, Sais, Sidi Mohamed Ben Abdellah University, Fez, Morocco

4_{Laboratoire d’Analyses Géo-Environnementales,}

d’Aménagement et de Développement Durable (LAGEA-DD), Vice Président Chargé des Affaires Académiques et Pédagogiques,

Université Sidi Mohamed Ben Abdellah, Fez, Morocco

5_{College of Education Curriculum}

and Instruction Department Kuwait University, Kuwait

*

These authors equally contributed to this study and should be considered as second co-authors.

† _{Corresponding Author: Dr. Nicola Luigi Bragazzi School of Public Health, Department of}

6_{Section of History of Medicine and Ethics,}

Department of Health Sciences (DISSAL), University of Genoa, Genoa, Italy

7_{School of Public Health, Department of Health Sciences (DISSAL),}

University of Genoa, Genoa, Italy

A

Identifying the variables that predict success at medical and allied health professions school is of major interest to university admissions committees given the cost of degree programmes and the social responsibility that schools have to graduate competent physicians and Healthcare personnel.

This research examined the criteria used by the Italian admission at Health schools to determine an applicant’s academic eligibility to attend the Health Professional degree programs.

In the literature there is general agreement that the admission process should include assessment of both cognitive and non-cognitive characteristics of applicants and several surveys have revealed that such assessment is common practice among health profession programs. In Italy these issues have been neglected.

Keywords: admission criteria, predictive validity, higher education

I

In recent years, the number of students enrolled in Italian university programs has shown a substantial increase. But the quality is likely to decrease instead of improving. Moreover, many degrees’ students are not well motivated and tend to lose their interest in their studies early. So the dropout and the delay in study progress are quite elevated.

Table 1. Overview of the aim of the stakeholders interested in a stringent admission criteria policy

Who (stakeholder) Which aim? University administrators

(Admission Committee)

forming the basis for recruiting strategies of a desired profile of students.

educating people for a profession in which personality and professional conduct play a central role.

Faculty ensuring adequate orientation, guidance and mentoring Medical and allied

Health Professions school applicants

knowing that the admission tools used are the most reliable assessment instruments

Professions and Health care systems

providing effective, professionally sighted and competent practitioners to make up the profession;

looking to select the correct candidates who will be able to uphold professional standards and raise the status of the profession in the eyes of the general public.

Health schools across the nation have used carefulness and creativity to develop and implement various screening measures with the final goal of predicting future clinical and professional performance.

In order to ensure the quality of research results, our participatory planning requires the involvement of concerned stakeholders.

In the space below, it can see the multiple stakeholders involved in this research and what they want to know about this question.

All stakeholders need to share their expertise in creating an evidence-based admissions process. And the goal of the Admission Committee who are responsible for selecting candidates for degree courses is to develop selecting tools that take into account the special level of knowledge, skills and personal characteristics that each profession requires.

In the literature there is general agreement that the admission process should include assessment of both cognitive and non-cognitive characteristics of applicants and several surveys have revealed that such assessment is common practice among health profession programs (Agho et al., 1998; Johnson & Edwards, 1991; Scott et al., 1995).

undertaken as stand-alone projects in their own right, especially where an area is complex or has not been reviewed comprehensively before.”

A

T

M

S

Nayer (1992) suggests that the aim of admission procedures is “to select students who will complete the university program and go into professional careers, do well in the program, perform creditably in professional practice and possess the traits of character and ethical values desired of a professional person.”

Selecting the best candidates for Healthcare Professionals degrees can be achieved when those who are selected have appropriate personality feature, such as caring for others, sense of responsibility and team orientation (Wharrad et al., 2003; Wilson, 1999).

So it is incumbent to select candidates from the total applicant pool who are most likely to succeed, not only as students in the program but also as Health Personnel in the future.

Despite the over the past 70 years debate about how best to select medical students, in the literature constantly appears the signal that some unsuitable candidate are still selecting (Adam et al., 2015; Powis, 2015).

The literature on medical school admissions consistently draws a distinction between cognitive and non-cognitive abilities. Broadly defined, the former refers to intellectual prowess, typically measured by grade point average (GPA) or performance on standardised tests of knowledge, such as the Medical College Admission Test (MCAT) in North America or the Graduate Australian Medical School Admission Test (GAMSAT) in Australia, the Undergraduate Medicine and Health Sciences Admission Test (UMAT) in Australia and New Zealand, the Medical Council of Canada qualifying examination part I and II (MCCQE) in Canada, the Health Professions Admission Test (HPAT) in Ireland, the BioMedical Admissions Test (BMAT) in the UK, Malaysia, Singapore, the Netherlands and Spain, and the UK Clinical Aptitude Test (UKCAT) in the UK.

Grade Point Average (GPA)

According to Kreiter (2007) the Grade Point Average has consistently shown statistically significant, practically relevant and positive predictive correlations with future performance.

Moreover, students from non-science backgrounds initially experience higher stress but ultimately perform equally well relative to their science background counterparts (Dickman et al., 1980; Yens & Stimmel 1982; Woodward & McAuley 1983; Neame et al., 1992; Koenig 1992; Huff & Fang 1999).

On the other hand, GPA well correlates with academic performance (Salem et al., 2016), even though needs to be adjusted considering race and ethnicity (Curtis et al., 2015).

Medical College Admissions Test: MCAT, UKCAT, HPAT, GAMSAT, UMAT, MCCQE and Beyond

The standardized Medical College Admission Test was first administered in 1928. Because of the changing knowledge, periodic reviews of standardized examinations are considered a best practice.

MCAT has not been not modified since 1991. In 2015, the Association of American Medical Colleges’ (AAMC) advisory committee was tasked with reviewing the examination and recommending changes.

The Committee’s final suggestions consider the following: (1) the content and the format of the new test, (2) essential resources to support changes, and (3) new data of integrity, service orientation, and other personal characteristics to be used in the process of student selection.

Regarding the recommendations for the content and format, the committee preserved the previous testing format that balanced the natural sciences, the social and behavioral sciences, and critical analysis and reasoning, and gave attention to concepts that future health personnel are likely to need.

The AAMC added a new section called Psychological, Social and Biological Foundations of Behavior, that measures candidates’ skill to understand sociocultural, biological and psychological stimuli on behavior and social relations as well as how people process emotion and stress.

It was also stressed the necessity of reading broadly to attend medical schools and of using scientific thinking, research, and statistics skills.

applicants’ personal characteristics may to be measured at national or local level using specific developed tools.

MCAT

The admission test, apportioned into four sections (Physical Sciences, Biological Sciences, Writing Sample, Verbal Reasoning), has consistently been shown to be statistically significant, practically relevant, with a positive predictive correlation with future performances (Donnon et al., 2007).

The multiple choice test is however the assessment tool more used in the world not only in a university environment but also in selection processes at work which involve a large number of candidates. Such a diffusion is due to several important advantages that only testing can guarantee:

 objectivity: the points scored by the candidates is not influenced by “personal” evaluations, but obtained through a standardized and computerized procedure that allows to make an objective assessment of evidence ensuring equal opportunities for all applicants;

 effectiveness: the tests, if properly designed and formulated, allow the identification of students who possess the knowledge, capabilities and attitudes required by the curriculum;

 rapidity: by automating the scanning of completed test documents it is possible to correct the tests of a large number of candidates and get their scores, accompanied by a comparative analysis, back to them on the same day as that of the test;

 efficiency: the tests constitute a low cost assessment tool because they can be distributed simultaneously to large groups of people, and especially because they can be corrected through automated procedures.

Also in Italy experience has shown that where the admission/evaluation test has been designed and operated with seriousness and professionalism, there is a strong correlation between the score obtained in the test and the results obtained in the course of the university studies.

In particular, authoritative research conducted by universities have shown that:

 students who obtain high marks in the admission test pass their exams with the higher grades and graduate with grades which are on average higher with respect to candidates positioned at the lower end of the list;

 the number of exam passed during the course is much higher when the grade achieved in the entrance test is greater;

 regulation of the access to universities through appropriate tests has made it possible to direct students toward degree programs that are most suitable to the aptitudes of individuals.

The same aforementioned studies have shown that the high school graduation mark and school results during the last school years, if used as a sole or dominant element in the assessment, possess a predictive capacity on the outcome of university education generally lower than those offered by the entrance tests.

The reason is mainly due to the fact that school grades are not able to discriminate between different types of school (high school, technical or vocational school, etc..) or between the different levels of quality of the individual institutions. In addition, a selective meritocratic entrance test, as well as playing a valuable support function for the orientation to higher education, represents an incentive for more motivated students to undertake a specific training course. The opportunity to be truly prepared to face the entrance examination is an opportunity for training that is also useful in university study curriculum. In fact, the idea that schools often fail to adequately perform the task of preparing students for university studies is increasingly shared. The opportunity provided by the entrance exams is therefore extremely valuable under this aspect.

In conclusion, the entrance test, far from discriminating against students based on factors related to family income or social environment, if well designed and operated, guarantees a merit-based selection criteria, thereby contributing to the achievement of a competitive system, which benefit the students themselves.

UKCAT

UKCAT in its original version is a 2-hour long test, comprising different sections: verbal reasoning, decision-making, quantitative reasoning and abstract reasoning.

Correlation between UKCAT and academic performance was demonstrated in a sample of 71 applicants at the University of Aberdeen Dental School between 2010 and 2014 (Foley and Hijazi, 2015).

Blackmur and colleagues (2016) used a modified version of UKCAT (15-minutes long) in a sample of 167 doctors and 26 lay people.

HPAT

HPAT was found to correlate with college results (Halpenny et al., 2010; Quinn et al., 2010), even though it has been criticized by some scholars (Henry, 2010).

GAMSAT

GAMSAT is a reasoning-based test, based on the item response theory. A number of studies has confirmed the validity of GAMSAT (Mercer et al., 2015; Puddey and Mercer, 2014).

Mercer and colleagues retrospectively reviewed academic performances in a span of 10 years and linking them with GAMSAT scores and found that GAMSAT performance has been stable and highly reliable, with significant variations in candidate performance related to age, gender, level and discipline of previous academic study and language background (Mercer et al., 2015). UMAT

BMAT

BMAT is 2-hour pen-and-paper test, divided into three section, namely: an aptitude and skills test (problem solving, critical thinking, data analysis, logic), a scientific knowledge and applications test, and a writing task. Emery and colleagues found some biases in BMAT scores and suggested to use BMAT as additional test together with interview performance (Emery et al., 2011). MCCQE

MCCQE, elaborated by the Medical Council of Canada (MCC) born in 1912, well correlated with academic performance in a number of studies. (Veale et al., 2014). In particular, in a sample of 367 students, the MCCQE-1 revealed good statistical properties, with R2 _{= 0.604 (Roy et al., 2016).}

However, some scholars are proposing to revise this questionnaire (Bowmer, 2015; Lougheed, 2016).

Multiple Mini-Interview

Small studies (Eva et al., 2004a, b, c; Eva et al., 2009; Reiter et al., 2007) confirm that the multiple mini-interview has consistently shown statistically significant, practically relevant and positive predictive correlations with future performance, such as OSCE performance (Oluwasanio et al., 2015).

For instance, Eva and colleagues (2004) have shown that, in a sample of forty five candidates to the Undergraduate MD program at McMaster University, MMI was able to predict pre-clerkship performance. In another group of 117 candidates, MMI showed a reliability of 0.65, whilst in a study carried out among 54 subjects a reliability coefficient of 0.78 could be found.

Harris and Owen (2007) confirmed the validity of MMI for student selection for the Australian National University Medical School in a sample of 115 candidates.

Heldebrand and colleagues (2016) found that students reporting a MMI score <4.5 were 3.5-times more likely to experience academic difficulties than students with a score of 5-6.

Indeed, most studies have been carried out in Western countries and few studies have been performed in non-Western settings, such as Korea, with a reliability coefficient of 0.791 in a sample of 84 applicants (Roh et al., 2009) or among Canadian aboriginals, yielding a reliability of 0.70 (Moreau et al., 2006).

Situational Judgement Test

Situational judgement test (SJT) assesses how you approach situations encountered in the workplace using hypothetical scenarios and simulations. Being situational, it measures non-cognitive skills and competency, like prosocial Implicit Trait Policies (ITPs), thus differing from traditional cognitive and academic abilities-based testing (Patterson et al., 2016). Usually SJTs are used in Europe. Lievens et al., (2005) demonstrated the operational validity of SJT in a cohort of 7197 medical and dental students in Belgium. Few studies have been conducted outside. For instance, Colbert-Getz and colleagues administered an ad hoc SJT to a sample of 489 applicants to the University of Utah Medical School.

However, some scholars have criticized SJT (Halliwell-Ewen et al., 2015; Najim et al., 2015; Sayma et al., 2016; Simon et al., 2015): Simon and colleagues, for example, found no correlation between SJT output and academic performance (Simon et al., 2015).

On the other hand, SJT has been successfully used also for selecting resident doctors (Patterson et al., 2016; Sureshkumar et al., 2016).

SJTs, indeed, seem to be effective according to some recently published meta-analyses and systematic reviews, but are time-consuming to be designed and implemented (Patterson et al., 2016).

As such, some scholars have proposed to use SJTs as additional tests, instead than basing exclusively on them.

Personal Interview (Traditional Interview)

2002). For medical school admissions interviews similar results have not been found (Albanese et al., 2003; Salvatori 2001).

Letters of Reference

Letters of references have been an integral part of the medical school admissions process and remains a common criteria in the candidate screening process (DeLisa et al., 1994; Berstein et al., 2002). There is, however, little evidence to support their effectiveness and their continued usage in the medical school admission process (Salvatori 2001). Many of the concerns that arise from the use of LORs as a selection tool can be attributed to its poor predictive validity (Kirchner and Holm 1997; Standridge et al., 1997) and poor reliability (Ross and Leichner 1984). LORs could be standardized (McCabe et al., 1989). On the other hand, studies by Ferguson and collaborators (Ferguson et al., 2003), Brothers and Wetherholt (2007) and Peskun et al., (2007) found letters of reference to be predictive of later performance (e.g., clinical performance in residency training).

Personal Statements

Research has indicated limited predictive value of the personal statement as a selection tool (McManus and Richards 1986; Dore et al., 2006; Hanson et al., 2007), even though other scholars have emphasized the wealth of data and information contained in personal statements (Ferguson et al., 2003; Wouters et al., 2014). However, there are few scientific evidences, with most reports being anecdotal rather than systematic (Albanese et al., 2003; Ferguson et al., 2002).

Personality Testing

Griffin and Wilson, 2012). Griffin and Wilson have found correlations between the Big Five Personality Questionnaire and the MMI (Griffin and Wilson, 2012).

However, the population of medical school aspirants tends to be far more homogeneous than those reported in human resource studies. Medical school applicant pools are homogeneously populated by high academic performers, so as to make the personality test unreliable in this setting. If it is true that personality traits exist and can differentiate among different medical students in correlating with academic performance and residency choice, on the other hand this effect, even though statistically significant, appears to be small (Bexelius et al., 2016; Schripsema et al., 2016).

Prior Academic Achievements

Several studies are consistent in indicating that prior academic achievements are good predictors of following academic performance (Gautam et al., 2012; Mercer et al., 2013).

I

N

I

P

Health Profession schools are obliged periodically to review their admissions criteria. Are they designed to select individuals with appropriate aptitudes or diversity, in such a way to be fairly inclusive and at the same time to be rigorous and selective? Are there more desirable applicants who fail to meet conventional criteria?

Identifying cognitive, non-cognitive and demographic variables that predict success at medical and allied health professions school is of major interest to university admissions committees given cost of degree programmes and the social responsibility schools have to graduate competent physicians and Healthcare personnel.

The traditional preadmission variables used in Italy are not addressing all the needs of medical school admissions committees grappling with the problem of identifying students with potential who did not have the educational opportunities afforded more fortunate students.

So, the university needs to identify variables that will predict success of all applicants not just those from privileged backgrounds. Having identified such students the school can provide additional support to ensure their subsequent success.

Secondly, the academic staff needs also to explore the predictive capacity of new admissions strategies such as the multiple mini interview strategy and determine its longer term utility.

Robust predictor variables of academic/clinical competence so as to measure success in a more professionally authentic manner have to be identified. These measures could then become the targets for selecting students for medical and allied Healthcare professions school.

C

The prediction of academic performance in a program of graduate education is a fundamental but not trivial task (Powis, 2015).

There is a considerable body of literature, mostly in the medical and nursing/midwifery press, related to the factors which are linked with, and are believed to affect, retention rates in health related courses. One of these is represented by the factors linked to ensure student have the right qualities (and a commitment to the future profession) and to select the most appropriate candidates for the profession.

The results do not allow to say that those who entered with a high admission test score (and probably will have high performance during the course) have also the skills to become a good health care personnel.

Current admissions criteria do not adequately address the social responsibilities and humanitarian goals of Healthcare education.

R

[1] Aber, C. S., Arathuzik, D. (1996). Factors associated with student success in a baccalaureate nursing programme within an urban public university. Journal of Nursing Education 35(6): 285–288.

[2] Adam J, Bore M, Childs R, Dunn J, Mckendree J, Munro D, Powis D. Predictors of professional behaviour and academic outcomes in a UK medical school: A longitudinal cohort study. Med Teach. 2015;37 (9):868-80.

[3] Adam J, Bore M, McKendree J, Munro D, Powis D. Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study. BMC

Med Educ. 2012 Aug 8;12:69.

[4] Adam J, Dowell J, Greatrix R. Use of UKCAT scores in student selection by U. K. medical schools, 2006-2010. BMC Med Educ. 2011 Nov 24;11:98.

[5] Agho, A. O., Mosley, B. W. & Smith-Paul, B. (1998). Use of interview in selecting students for occupational therapy programs. American

Journal of Occupational Therapy 52: 592–594.

[6] Albanese, M. A., Snow, M. H., Skochelak, S. E., Huggett, K. N., & Farrell, P. M. (2003). Assessing personal qualities in medical school admissions. Academic Medicine, 78(3), 313–321.

[7] Aldous CJ, Leeder SR, Price J, Sefton AE, Teubner JK. A selection test for Australian graduate-entry medical schools. Med J Aust. 1997 Mar 3;166(5):247-50.

[8] Bajwa NM, Yudkowsky R, Belli D, Vu NV, Park YS. Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study. Adv Health Sci Educ Theory

Pract. 2016 Apr 23. [Epub ahead of print].

[9] Barrick, M. R., & Mount, M. K. (1991). The Big 5 personality dimensions and job performance: A metaanalysis. Personnel

Psychology, 44, 1–26.

[10] Bath M. Regulation of preparation courses for the Situational Judgement Test. Med Teach. 2016 Jul;38(7):752.

[12] Bodger O, Byrne A, Evans PA, Rees S, Jones G, Cowell C, Gravenor MB, Williams R. Graduate entry medicine: selection criteria and student performance. PLoS One. 2011;6(11):e27161.

[13] Bowles N. 1995. Methods of nurse selection: a review. Clinical

Management Education, 4, 25–29.

[14] Brothers, T. E., & Wetherholt, S. (2007). Importance of the faculty interview during the resident application process. Journal of Surgical

Education, 64(6), 378–385.

[15] Burg FD, Croll SR, Ruff GE, Stemmler EJ. Competency requirements. A new approach to medical school admissions. JAMA. 1988 Jan 15; 259(3):389-91.

[16] Burkhardt JC, DesJardins SL, Teener CA, Gay SE, Santen SA. Enrollment Management in Medical School Admissions: A Novel Evidenced-Based Approach at One Institution. Acad Med. 2016 Apr 12. [Epub ahead of print].

[17] Cassidy J. UKCAT among the pigeons. BMJ. 2008 Mar 29;336 (7646):691-2.

[18] Cleland JA, French FH, Johnston PW; Scottish Medical Careers Cohort Study Group. A mixed-methods study identifying and exploring medical students’ views of the UKCAT. Med Teach. 2011;33(3):244-9.

[19] Coates H. Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ. 2008 Oct;42(10):999-1006.

[20] Crockford PM, Gupta DM, Grace MG. Requirements for admission to medical school: how many years of university study are necessary?

CMAJ. 1995 Dec 1;153(11):1595-600.

[21] DeLisa, J. A., Jain, S. S., & Campagnolo, D. I. (1994). Factors used by physical medicine and rehabilitation residency training directors to select their residents. American Journal of Physical Medicine and

Rehabilitation, 73(3), 152–156.

[22] Dennehy T, Kelly M, O’Flynn S. General practitioners’ perspectives on revised entry and selection methods to medicine and the HPAT. Ir Med

J. 2013 Apr;106(4):113-5.

[23] de Silva NR, Pathmeswaran A, de Silva HJ. Selection of students for admission to a medical school in Sri Lanka. Ceylon Med J. 2004 Sep;49(3):81-5.

[25] Dickman, R. L., Sarnacki, R. E., Schimpfhauser, F. T., & Katz, L. A. (1980). Medical students from natural science and nonscience undergraduate backgrounds. Similar academic performance and residency selection. Journal of the American Medical Association, 243 (24), 2506–2509.

[26] Donnon, T., Paolucci, E. O., & Violato, C. (2007). The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research.

Academic Medicine, 82(1), 100–106.

[27] Dore, K. L., Hanson, M., Reiter, H. I., Blanchard, M., Deeth, K., & Eva, K. W. (2006). Medical school admissions: Enhancing the reliability and validity of an autobiographical screening tool. Academic Medicine, 81 (10, Suppl), S70–S73.

[28] Edwards D, Friedman T, Pearce J. Same admissions tools, different outcomes: a critical perspective on predictive validity in three undergraduate medical schools. BMC Med Educ. 2013 Dec 27;13:173. [29] Ehrenfeld M., Tabak N. (2000). Value of admission interviews in

selecting of undergraduate nursing students. Journal of Nursing

Management, 8, 101–106.

[30] Emery JL, Bell JF, Vidal Rodeiro CL. The BioMedical Admissions Test for medical student selection: issues of fairness and bias. Med Teach. 2011;33(1):62-71.

[31] Eva KW, Macala C. Multiple mini-interview test characteristics: ‘tis better to ask candidates to recall than to imagine. Med Educ. 2014 Jun;48(6):604-13.

[32] Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004). The ability of the multiple mini-interview to predict preclerkship performance in medical school. Academic Medicine, 79 (10, Suppl), S40–S42.

[33] Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009). Predictive validity of the multiple mini-interview for undergraduate and postgraduate admissions decisions. Medical

Education (accepted for publication).

[35] Fang JT, Ko YS, Chien CC, Yu KH. Assessing medical students’ performance in core competencies using multiple admission programs for colleges and universities: from the perspective of multi-source feedback. Biomed J. 2013 Jul-Aug;36(4):188-97.

[36] Ferdjani JK, Patel AB, Patel SR. One-dimensional learning: detrimental to the Situational Judgement Test? Br J Gen Pract. 2012 Oct; 62 (603):520.

[37] Finucane P, Arnett R, Johnson A, Waters M. Graduate medical education in Ireland: a profile of the first cohort of students. Ir J Med

Sci. 2008 Mar;177(1):19-22.

[38] Gardner SP, Roberts-Thomson KF. Predicting high achievers in the University of Adelaide, Australia, Bachelor of Oral Health program, 2002-09. J Dent Educ. 2012 Dec;76(12):1646-56.

[39] Garrud P, Yates J. Profiling strugglers in a graduate-entry medicine course at Nottingham: a retrospective case study. BMC Med Educ. 2012 Dec 18;12:124.

[40] Gautam AP, Paudel BH, Agrawal CS, Niraula SR, Dalen JV. Examination of relationship of scores obtained in grades 10 and 12 with the entry and success in undergraduate medical education. Kathmandu

Univ Med J (KUMJ). 2012 Jan-Mar;10(37):66-71.

[41] Gliatto P, Leitman IM, Muller D. Scylla and Charybdis: The MCAT, USMLE, and Degrees of Freedom in Undergraduate Medical Education.

Acad Med. 2016 May 31. [Epub ahead of print]

[42] Glossop, C., 2002. Student nurse attrition: use of an exit interview procedure to determine students’ leaving reasons. Nurse Education Today 22, 375–386.

[43] Griffin BN. Predictive validity of the UMAT for medical students’ academic performance. Med J Aust. 2011 Jul 18;195(2):102.

[44] Griffin B, Carless S, Wilson I. The Undergraduate Medical and Health Sciences Admissions Test: what is it measuring? Med Teach. 2013 Sep;35(9):727-30.

[45] Griffin B, Hu W. The interaction of socio-economic status and gender in widening participation in medicine. Med Educ. 2015 Jan;49(1):103-13. [46] Griffin B, Wilson I. Associations between the big five personality

factors and multiple mini-interviews. Adv Health Sci Educ Theory Pract. 2012 Aug;17(3):377-88.

[48] Halliwell-Ewen MJ, Lee C, Gopal DP. Deanery placement and the Situational Judgement Test. Adv Med Educ Pract. 2015 Nov 23;6:631-2. [49] Halpenny D, Cadoo K, Halpenny M, Burke J, Torreggiani WC. The

Health Professions Admission Test (HPAT) score and leaving certificate results can independently predict academic performance in medical school: do we need both tests? Ir Med J. 2010 Nov-Dec;103(10):300-2. [50] Hanson, M. D., Dore, K. L., Reiter, H. I., & Eva, K. W. (2007). Medical

school admissions: Revisiting the veracity and independence of completion of an autobiographical screening tool. Academic Medicine, 82, S8–S11.

[51] Hanson MD, Moineau G, Kulasegaram KM, Hammond R. Is Canada Ready for Nationwide Collaboration on Medical School Admissions Practices and Policies? Acad Med. 2016 Jul 5. [Epub ahead of print]. [52] Harris BH, Walsh JL, Lammy S. UK medical selection: lottery or

meritocracy? Clin Med (Lond). 2015 Feb;15(1):40-6.

[53] Heldenbrand SD, Flowers SK, Bordelon BJ, Gubbins PO, O’Brien C, Stowe CD, Martin BC. Multiple Mini-Interview Performance Predicts Academic Difficulty in the PharmD Curriculum. Am J Pharm Educ. 2016 Mar 25;80(2):27.

[54] Henry M. Unfair discrimination under UKCAT and HPAT. Ir Med J. 2010 May;103(5):154.

[55] Houltram, B., 1996. Entry age, entry mode and academic performance on a project 2000 common foundation programme. Journal of Advanced

Nursing 23, 1089–1097.

[56] Huff, K. L., & Fang, D. (1999). When are students most at risk of encountering academic difficulty? A study of the 1992 matriculants to US medical schools. Academic Medicine, 74(4), 454–460.

[57] Husbands A, Rodgerson MJ, Dowell J, Patterson F. Evaluating the validity of an integrity-based situational judgement test for medical school admissions. BMC Med Educ. 2015 Sep 2;15:144.

[58] James D, Yates J, Nicholson S. Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study. BMJ. 2010 Feb 16;340:c478.

[60] Johnson, E. K. & Edwards, J. C. (1991). Current practices in admission interviews at U. S. medical schools. Academic Medicine 66: 408–412. [61] Katelaris AG. What’s the matter with UMAT? Med J Aust. 2011 Apr

4;194(7):330.

[62] Kay-Lambkin F, Pearson SA, Rolfe I. The influence of admissions variables on first year medical school performance: a study from Newcastle University, Australia. Med Educ. 2002 Feb;36(2):154-9. [63] Kelly ME, Gallagher N, Dunne FP, Murphy AW. Views of doctors of

varying disciplines on HPAT-Ireland as a selection tool for medicine.

Med Teach. 2014 Sep;36(9):775-82.

[64] Kiessling C, Bauer J, Gartmeier M, Iblher P, Karsten G, Kiesewetter J, Moeller GE, Wiesbeck A, Zupanic M, Fischer MR. Development and validation of a computer-based situational judgement test to assess medical students’ communication skills in the field of shared decision making. Patient Educ Couns. 2016 Jun 14. pii: S0738-3991(16)30270-1. doi: 10.1016/j.pec.2016.06.006. [Epub ahead of print].

[65] Kirchner, G. L., & Holm, M. B. (1997). Prediction of academic and clinical performance of occupational therapy students in an entry-level master’s program. The American Journal of Occupational Therapy, 51, 775–779.

[66] Koczwara A, Patterson F, Zibarras L, Kerrin M, Irish B, Wilkinson M. Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection. Med Educ. 2012 Apr;46(4):399-408. [67] Koenig, J. A. (1992). Comparison of medical school performances and

career plans of students with broad and with science-focused premedical preparation. Academic Medicine, 67(3), 191–196.

[68] Kreiter, C. D., & Kreiter, Y. (2007). A validity generalization perspective on the ability of undergraduate GPA and the medical college admission test to predict important outcomes. Teaching and Learning in

Medicine, 19(2), 95–100.

[69] Krumm S, Lievens F, Hüffmeier J, Lipnevich AA, Bendels H, Hertel G. How “situational” is judgment in situational judgment tests? J Appl

Psychol. 2015 Mar;100(2):399-416.

[70] Kumwenda B, Dowell J, Husbands A. Is embellishing UCAS personal statements accepted practice in applications to medicine and dentistry?

Med Teach. 2013 Jul;35(7):599-603.

[72] Laurence CO, Zajac IT, Lorimer M, Turnbull DA, Sumner KE. The impact of preparatory activities on medical school selection outcomes: a cross-sectional survey of applicants to the University of Adelaide Medical School in 2007. BMC Med Educ. 2013 Dec 1;13:159.

[73] Lievens F, Buyse T, Sackett PR. The operational validity of a video-based situational judgment test for medical college admissions: illustrating the importance of matching predictor and criterion construct domains. J Appl Psychol. 2005 May;90(3):442-52.

[74] Lucieer SM, Stegers-Jager KM, Rikers RM, Themmen AP. Non-cognitive selected students do not outperform lottery-admitted students in the pre-clinical stage of medical school. Adv Health Sci Educ Theory

Pract. 2016 Mar;21(1):51-61.

[75] Luschin-Ebengreuth M, Dimai HP, Ithaler D, Neges HM, Reibnegger G. Situational judgment test as an additional tool in a medical admission test: an observational investigation. BMC Res Notes. 2015 Mar 14;8:81. [76] Lynch B, Mackenzie R, Dowell J, Cleland J, Prescott G. Does the

UKCAT predict Year 1 performance in medical school? Med Educ. 2009 Dec;43(12):1203-9.

[77] Magennis T, Mitchell J. University entry scores as a predictor of academic performance in a health information management program.

Health Inf Manag. 1998 Jun-Aug;28(2):57-61.

[78] McAndrew R, Ellis J, Valentine RA. Does a selection interview predict year 1 performance in dental school? Eur J Dent Educ. 2016 Feb 22. doi: 10.1111/eje.12188. [Epub ahead of print].

[79] McManus IC, Dewberry C, Nicholson S, Dowell JS, Woolf K, Potts HW. Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies. BMC Med. 2013 Nov 14;11:243.

[80] McManus, I. C., & Richards, B. (1986). Prospective study of performance of medical students during preclinical years. British

Medical Journal, 293, 124–127.

[81] Mercer A, Crotty B, Alldridge L, Le L, Vele V. GAMSAT: A 10-year retrospective overview, with detailed analysis of candidates’ performance in 2014. BMC Med Educ. 2015 Mar 5;15:31.

[82] Najim M, Rabee R, Sherwani Y, Ahmed M, Ashraf M, Al-Jibury O, Najim R. The situational judgement test: a student’s worst nightmare.

[83] Nayer, M. (1992). Admission criteria for entrance to physiotherapy schools: How to choose among many applicants. Physiotherapy Canada 44: 41–46.

[84] Neame, R. L., Powis, D. A., & Bristow, T. (1992). Should medical students be selected only from recent school-leavers who have studied science? Medical Education, 26(6), 433–440.

[85] Nedjat S, Majdzadeh R, Rashidian A. Graduate entry to medicine in Iran. BMC Med Educ. 2008 Oct 11;8:47.

[86] O’Flynn S, Fitzgerald T, Mills A. Modelling the impact of old and new mechanisms of entry and selection to medical school in Ireland: who gets in? Ir J Med Sci. 2013 Sep;182(3):421-7.

[87] Ofori, R., 2000. Age and ‘type’ of domain specific entry qualifications as predictors of student nurses’ performance in biological, social and behavioural sciences in nursing assessments. Nurse Education Today 20, 298–310.

[88] Patel A, Philip A. The Foundation Programme application: an alternative to the Situational Judgement Test. Adv Med Educ Pract. 2015 Oct 14;6:589-90.

[89] Patterson F, Zibarras L, Ashworth V. Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. Med Teach. 2016;38(1):3-17.

[90] Pau A, Chen YS, Lee VK, Sow CF, De Alwis R. What does the multiple mini interview have to offer over the panel interview? Med Educ Online. 2016 Feb 11;21:29874.

[91] Peskun, C., Detsky, A., & Shandling, M. (2007). Effectiveness of medical school admissions criteria in predicting residency ranking four years later. Medical Education, 41(1), 57–64.

[92] Plaisant O, Stephens S, Apaydin N, Courtois R, Lignier B, Loukas M, Moxham B. Medical students’ attitudes towards science and gross anatomy, and the relationship to personality. J Anat. 2014 Mar;224 (3):261-9.

[93] Poole P, Shulruf B. Shaping the future medical workforce: take care with selection tools. J Prim Health Care. 2013 Dec 1;5(4):269-75. [94] Powis D. Selecting medical students: An unresolved challenge. Med

Teach. 2015 Mar;37(3):252-60.

[95] Puddey IB, Mercer A. Socio-economic predictors of performance in the Undergraduate Medicine and Health Sciences Admission Test (UMAT).

[96] Puddey IB, Mercer A. Predicting academic outcomes in an Australian graduate entry medical programme. BMC Med Educ. 2014 Feb 15;14:31. [97] Puddey IB, Mercer A, Andrich D, Styles I. Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT). BMC Med Educ. 2014 Mar 12;14:48. [98] Quinn A, Corrigan MA, Broderick J, McEntee G, Hill AD. A

comparison of performances of consultant surgeons, NCHDs and medical students in a modified HPAT examination. Ir Med J. 2010 Jun;103(6):172-3.

[99] Rahbar MH, Vellani C, Sajan F, Zaidi AA, Akbarali L. Predictability of medical students’ performance at the Aga Khan University from admission test scores, interview ratings and systems of education. Med

Educ. 2001 Apr;35(4):374-80.

[100] Ramachandran S, Venkatesh H. A call for greater transparency in the evidence base supporting the BMAT. Med Teach. 2015 Mar;37(3):302. [101] Razali SM. Medical school entrance and career plans of Malaysian

medical students. Med Educ. 1996 Nov;30(6):418-23.

[102] Reeves RE, Vishwanatha JK, Yorio T, Budd M, Sheedlo HJ. The post-baccalaureate premedical certification program at the University of North Texas Health Science Center strengthens admission qualifications for entrance into medical school. Acad Med. 2008 Jan;83(1):45-51. [103] Reiter, H. I., Eva, K. W., Rosenfeld, J., & Norman, G. R. (2007).

Multiple mini-interviews predict clerkship and licensing examination performance. Medical Education, 41(4), 378–384.

[104] Rockstuhl T, Ang S, Ng KY, Lievens F, Van Dyne L. Putting judging situations into situational judgment tests: evidence from intercultural multimedia SJTs. J Appl Psychol. 2015 Mar;100(2):464-80.

[105] Ross, C. A., & Leichner, P. (1984). Criteria for selecting residents: A reassessment. Canadian Journal of Psychiatry, 29, 681–686.

[106] Roy B, Ripstein I, Perry K, Cohen B. Predictive value of grade point average (GPA), Medical College Admission Test (MCAT), internal examinations (Block) and National Board of Medical Examiners (NBME) scores on Medical Council of Canada qualifying examination part I (MCCQE-1) scores. Can Med Educ J. 2016 Mar 31;7(1):e47-56. [107] Salvatori, P. (2001). Reliability and validity of admissions tools used to

select students for the health professions. Advances in Health Sciences

Education, 6, 159–175.

[109] Sartania N, McClure JD, Sweeting H, Browitt A. Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study. BMC Med Educ. 2014 Jun 11;14:116. [110] Sayma M, Jamshaid S, Kerwat D, Saleh D, Ahmed A, Oyewole F, Wahid AS. Going beyond the situational judgment test: the true impact of the educational performance measure, publications, and extra-degree performance on the foundation program application. Adv Med Educ

Pract. 2016 Feb 9;7:63-4.

[111] Schripsema NR, van Trigt AM, van der Wal MA, Cohen-Schotanus J. How Different Medical School Selection Processes Call upon Different Personality Characteristics. PLoS One. 2016 Mar 9;11(3):e0150645. [112] Scott, A. H., Chase L. M., Lefkowitz, R., Morton-Rias, D., Chambers,

C., Joe, J., Holmes, G. & Bloomberg, S. (1995). A national survey of admissions criteria and processes in selected allied health professions.

Journal of Allied Health 24: 95–106.

[113] Sharma N. Medical students’ perceptions of the situational judgement test: a mixed methods study. Br J Hosp Med (Lond). 2015 Apr;76 (4):234-8.

[114] Sharma N, Morgan M, Walsh K. Faculty perception of the medical student situational judgement test - Summary of findings. Med Teach. 2016 May;38(5):529.

[115] Shulruf B, Poole P, Wang GY, Rudland J, Wilkinson T. How well do selection tools predict performance later in a medical programme? Adv

Health Sci Educ Theory Pract. 2012 Dec;17(5):615-26.

[116] Simon E, Walsh K, Paterson-Brown F, Cahill D. Ranking and situational judgement test: response. Clin Teach. 2015 Dec;12(6):438.

[117] Simpson PL, Scicluna HA, Jones PD, Cole AM, O’Sullivan AJ, Harris PG, Velan G, McNeil HP. Predictive validity of a new integrated selection process for medical school admission. BMC Med Educ. 2014 Apr 23;14:86. doi: 10.1186/1472-6920-14-86.

[118] Sladek RM, Bond MJ, Frost LK, Prior KN. Predicting success in medical school: a longitudinal study of common Australian student selection tools. BMC Med Educ. 2016 Jul 22;16(1):187.

[119] Stacey DG, Kurunathan TM. Noncognitive Indicators as Critical Predictors of Students’ Performance in Dental School. J Dent Educ. 2015 Dec;79(12):1402-10.

[121] Stimmel BD. The educational process at the Mount Sinai School of Medicine: the first 25 years. Mt Sinai J Med. 1989 Oct;56(5):375-82. [122] Sureshkumar P, Roberts C, Clark T, Jones M, Hale R, Grant M. Factors

related to doctors’ choice of rural pathway in general practice specialty training. Aust J Rural Health. 2016 Jul 5.

[123] Tiffin PA, McLachlan JC, Webster L, Nicholson S. Comparison of the sensitivity of the UKCAT and A Levels to sociodemographic characteristics: a national study. BMC Med Educ. 2014 Jan 8;14:7. [124] Turner R, Nicholson S. Can the UK Clinical Aptitude Test (UKCAT)

select suitable candidates for interview? Med Educ. 2011 Oct; 45(10):1041-7.

[125] Van Rooyen P, Dixon A, Dixon G, Wells C. Entry criteria as predictor of performance in an undergraduate nursing degree programme. Nurse

Educ Today 2006; 26:593-600.

[126] Wharrad, H. J., Chapple, M., Price, N., 2003. Predictors of academic success in a Bachelor of Nursing course. Nurse Education Today 23, 246–254.

[127] Wiesner, W., & Cronshaw, S. (1988). A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61, 275–290.

[128] Wilkinson D, Casey MG, Eley DS. Removing the interview for medical school selection is associated with gender bias among enrolled students.

Med J Aust. 2014 Feb 3;200(2):96-9.

[129] Wilkinson D, Zhang J, Byrne GJ, et al., Medical school selection criteria and the prediction of academic performance. Evidence leading to change in policy and practice at the University of Queensland. Med J Aust 2008; 188: 349-354.

[130] Wilkinson TM, Wilkinson TJ. Preparation courses for a medical admissions test: effectiveness contrasts with opinion. Med Educ. 2013 Apr;47(4):417-24.

[131] Wilson T., 1999. A student selection method and predictors of success in a graduate nursing program. Journal of Nursing Education 38 (4), 183– 193.

[132] Woodward, C. A., & McAuley, R. G. (1983). Can the academic background of medical graduates be detected during internship?

[133] Wouters A, Bakker AH, van Wijk IJ, Croiset G, Kusurkar RA. A qualitative analysis of statements on motivation of applicants for medical school. BMC Med Educ. 2014 Sep 23;14:200.

[134] Yates J, James D. The UK Clinical Aptitude Test and clinical course performance at Nottingham: a prospective cohort study. BMC Med

Educ. 2013 Feb 26;13:32.

[135] Yens, D. P., & Stimmel, B. (1982). Science versus nonscience undergraduate studies for medical school: A study of nine classes.