Method

Phase One objective: to identify which specialty trainees have difficulty progressing through their annual review and the reasons why.

To help achieve the objective of this scoping exercise the following methods were adopted:

3. Meetings with key members of staff at HEE regional office to investigate what data and information was held about trainees who have had extended periods of training and the reasons.

4. Retrospective observational study. Access to existing HEE regional office data to help identify which trainees have extended periods of training.

3.2.1 Data Collection

3.2.1.1 Information Gathering

Introductory meetings with key staff at HEE regional office were arranged to explain the nature of the project and to request assistance with data

collection. Several HEE regional office staff members were involved in this project, facilitating access to data, assisting with data collection and sharing insight into issues relating to this phase of the research.

3.2.1.2 Ethics

Ethical approval was successfully sought through the Durham University School of Medicine, Pharmacy and Health ethics subcommittee (appendix 2). A National Research Ethics Service (NRES) review was not required for this

62

research as it did not require the participation of patients. However, the researcher was required to hold a Disclosure and Barring Service (DBS) check and a Confidentiality Agreement was put in place between the regional HEE office and Durham University, signed by both parties to allow access to anonymised data.

3.2.1.3 Database information

Data from a five-year period were interrogated in one HEE regional office to establish if there were any patterns associated with characteristics such as gender, age and country of medical school qualification, which related to trainees receiving adverse ARCP or Record of in Training Assessment (RITA) outcomes (see Table 3) which presents the range of outcomes a trainee may receive in their ARCP or RITA). Analyses also explored why an adverse ARCP outcome was received. A trainee could receive an adverse outcome for a single reason or for multiple reasons. A list of ‘U codes’ - reasons for receiving an adverse outcome - are outlined in table 4.

Anonymised data were taken from one HEE regional office’s ARCP/RITA outcome database for the five-year period from August 2009 to August 2013 on all trainees within that HEE regional office who were training within that period. Trainees were Foundation Programme doctors, Specialty doctors (including Core trainees and General Practitioners) in one geaographical training region.

63

3.2.2 Data Cleaning and Analysis

Data was extracted in the form of three Excel spreadsheets covering post history, staff details and assessment details. The Excel spreadsheets were password protected and anonymised of any identifiable data.

The data required extensive cleaning before the analysis of the data could begin. In particular it needed to be transformed so that each row represented the multiple ARCP results of one individual. Advice and help was sought from a quantitative expert from Durham University to ensure that this process was undertaken correctly.

Individuals in three spreadsheets had a unique person identifier, which could be matched up across the three spreadsheets to enable merging into one spreadsheet. There were multiple entries for each trainee on the assessment and post history spreadsheets to accommodate the number of posts held and RITA/ARCP assessments carried out by the trainees (n=85,000 records in total). The three Excel spreadsheets were transferred into an Access database and a table of all individual records was merged using the unique identifiers with the assessment records and post history with assessment records.

The data was cleaned and re-coded before analysis could begin. If there was a blank in a field, then ‘Not Stated’ was assigned to that field. To overcome the changes between the previous annual appraisal systems – RITA, which some trainees were still using, the RITA outcomes were mapped onto the ARCP outcomes for ease of analysis (RITA and ARCP codes did not map directly onto each other). Therefore, in the outcomes field a RITA D or E outcome were replaced with an ARCP outcome three/four (fail) (Gold Guide,

64

2016) See table 3 above for a breakdown of ARCP and RITA outcomes). At the start of this thesis I was interested in trainees having to extend their training. Therefore, the focus of the scoping exercise included outcomes three (extended training) and four (leave the training programme). However, the focus of the study was later broadened to include trainees who received an outcome two (targeted training), and these trainees were interviewed in Phase Three of this thesis. The data was then transferred from Access back into Excel for further data cleaning. Data was then re-coded again for ease of analysis.

There were several variables on the database which were excluded in the analysis because much of the data was missing. These were: ethnicity, specialty, grade, whether they were full time or less than full time. In addition, trainees had received more than one ARCP outcome over the five year period that data was analysed but the dependent vairiable (pass/fail) is based upon the last ARCP outcome the individual trainee had received. The effect of trainee age was explored in the scoping exercise, however there is no accepted standard definition of young and mature medical student (Pyne and Shlomo, 2015).The following age categories were

selected and the data was re-coded into three age categories: 20-30, 31-40, 41-plus, which ensured sufficient numbers per cell for the chi square

analyses. Age was treated as a continuous variable for the logistic

regression analysis. Country of medical school graduation was also re-coded so that there were three main categories: UK, EU and IMG, for ease of

analysis and to ensure the data was non-identifiable. Due to the small

65

Data was analysed using Excel, R and SPSSv23 using a combination of descriptive statistics (percentages and frequencies) and inferential statistics (chi square, binary logistic regression). Help and advice were sought for the logistic regression modelling from colleagues.