THINKING OUTSIDE THE BO

The second objective of this thesis was to explore whether approaches from other domains can be applied to medical student selection. The investigated approached included: the use of self-regulated learning skills of students in selection, the expert performance approach, and the relationship between a predictor (i.e. a selection method) and a criterion (i.e. a measure of performance).

According to the field of personnel selection, self-regulated learning skills, and especially self-efficacy and self-monitoring, positively relate to job performance. These skills help people to learn more efficiently, and to stay more easily up to date on all the developments in their fields. Only, these skills are not fixed but develop gradually. Hence, before applying these measures to medical school selection procedures, the potential development of these skills during medical school should be investigated. The cross-sectional study described in Chapter 3 examines the development of self- regulated learning in the pre-clinical stages of medical school, i.e. between year one and year three. In addition, since medical schools also want to include students who perform academically well, the relation between self-regulated learning skills and academic performance is investigated. The main finding here was that the self-regulated learning skills, except for reflection, did not develop between the first and third year of medical school. Apart from this, effort, monitoring, and reflection were slightly related to academic performance in the first year, and only one skill, effort, was related to performance in the third year.

Some researchers argued that self-regulated learning skills develop differently depending on the learning environment. The study reported in Chapter 4 hence focused on the development of self-regulated learning skills in two distinct medical curricula in Brazil: a problem-based and a lecture-based curriculum. Surprisingly, the main finding here was that in the lecture-based curriculum, some self-regulated learning skills, i.e. planning, monitoring, and evaluation, decreased during medical school. In the problem- based curriculum, sixth-semester students were slightly better than second-semester students in reflection, but showed less effort than second-semester students. Though these findings may be influenced by culture, they do indicate that the environment plays a major role in the development of self-regulated learning skills.

Another approach explored in Chapter 5 is the expert performance perspective. According to this perspective, one will not reach top levels without prolonged, structured training, also labelled as deliberate practice. The expert performance approach usually attempts to describe performance under standardized conditions and representative tasks wherein superior performance can be investigated. The complex character of the medical field makes it impossible to compare performance of medical doctors, and consequently little research has been conducted on the role of deliberate practice in the medical field. Residents, however, are assessed during their training, which enabled us to investigate whether behavioural characteristics fit in the concept of deliberate practice related to residency performance. Using a questionnaire to measure their behaviour and supervisor judgement as indicator of performance, this study revealed that those

Ch

ap

ter 8

Summ

ary

residents who performed in the top two tertiles, more frequently restudied the literature, asked for feedback, and acknowledged that they made mistakes. The latter is necessary to receive appropriate feedback. These findings indicate that deliberate practice is, to some extent, able to explain differences in performance of residents. However, further research is needed to investigate whether these behavioural differences are already present at time of application for medical school, and could be used in medical school selection to identify promising future medical doctors.

Subsequently, in Chapter 6, the criterion-predictor relation is highlighted. In this study, the relation between pre-training variables and performance during surgical residency training was examined. In order to investigate whether selection methods are able to predict how well someone will perform as a medical doctor, a criterion of good performance as a medical doctor is required. As residents, contrary to medical doctors, are assessed, their medical performance is, for now, the most optimal criterion available that can be used to measure the effectiveness of a selection method. In this study, biographical data, secondary and medical school performance, academic credentials and clinical work experience, were related to scores on in-training knowledge tests and to global performance scores. While mean pre-clinical grade related to both indictors of success, males only outperformed females on the knowledge test. As regards the global performance scores, aside from the pre-clinical grade, residents with an Atheneum secondary-school diploma, higher secondary-school mathematics grades and less work experience received higher scores. Selection on some of these variables is not allowed and a few other measures are not available at time of application for medical school, but secondary-school mathematics grades are available and its use is permitted. It would therefore be interesting to measure the relation between this variable and performance in other training programmes, to determine whether these mathematics grades should be valued more in medical school selection.

Finally, in Chapter 7, a general discussion on the studies described in this thesis is provided. The discussion starts with a short summary of the main findings and considerations for medical schools and medical school selection committees. These considerations include 1) the advice to use of cognitive selection methods, 2) that self-regulated learning skills can be used in medical school selection but it is perhaps more important to help students develop their self-regulated learning skills during medical school, or at least, make sure they do not decrease, 3) to apply the elements of deliberate practice in residency training as this will be beneficial to the residents’ performance, and to investigate the presence of these behavioural characteristics at time of application to medical school and 4) to investigate the relation between pre-training variables and residency performance in other training programmes besides surgical residency training, in order to determine whether secondary school mathematics grades should be valued more in medical school selection. Apart from this, the remaining challenges in medical school selection as well as considerations for future research are provided.

SAMENVATTING