Round two

My external supervisor (LBD) and I divided the 156 statements into three categories: Knowledge, skills and attitudes (n=115), organisational outcomes (n=8) and negative outcomes (n=33). Statements were presented alongside a seven point Likert-type scale, regarding agreement as to whether each statement should be ‘considered a ‘core outcome’ of international placements that should be measured in a toolkit’. The scale used the following numbers to represent agreement: 1=Strongly Disagree, 2=Disagree, 3=Slightly Disagree, 4=No Preference, 5=Slightly Agree, 6=Agree, 7= Strongly Agree. For emphasis the phrase ‘core outcome’ was presented in bold and the definition was repeated in

numerous emails, instructions and synopsis. A ‘core outcome’ was defined in the following way:

“A core outcome is something that is common, important and applicable across a wide range of settings. It can be a benefit or cost, but it must be something that would be more likely to happen to an individual on international placement rather than somebody working in the UK”

For each round participants had 14 days to respond. However, as this initial questionnaire was particularly long some participant’s requested an extension of the deadline by 10 days. Email reminders were sent to invitees frequently.

6.2.3.3. Round three

The statements with at least 70% consensus in the previous round were retained and not re- presented to the group. In round three, stakeholders that completed round two were

presented with the much smaller group of non-consensus statements and an anonymised report of the comments gathered in round two. Participants were asked to use the same Likert scale and reconsider their answers from round two (displayed) in light of the group median and the comments. Participants were given 14 days to answer but some requested a 2-day extension.

6.2.3.4. Round four

Any statements with at least 70% consensus in round three were retained; therefore participants were presented with an even smaller list of statements. Participants who had not responded in round three (but had in round two) were invited to re-join the study, as many stakeholders were working internationally and had limited internet access at certain

154

periods of the study. In round four, the expressions of some statements were changed in light of participant comments to increase clarity.

Box 3: The three questions presented to stakeholders in the Delphi

6.2.3. Analysis

The COS that is developed will then be compared to the current knowledge base

developed in a systematic review by Jones et al. (13). The current outcomes will be applied to the broad domains generated in the systematic review to see the spread of items within each domain and any items that fit into more than 1 domain or no domain. The domains are: (communication and teamwork, clinical skills, management skills, patient experience and dignity, policy, academic skills and personal satisfaction & interest). A Wilcoxon ranked sums test will be conducted on the results between rounds to show the changes in opinions between rounds.

6.3. Results

6.3.1. Participants

51 participants attended the round one workshop across all of the stakeholder’s groups. In total, 259 participants were invited to the online Delphi, 78 accepted. The response rates throughout the rounds remained high, however there was a small amount of attrition (22%): round two n=58, round three n=49, round four n=45. The stakeholders did not form a homogeneous group, nor fit into single defined categories (see appendix 11). More than half of the participants were involved in global health policy and a third of the

participants had volunteered themselves in a healthcare role. Box 3

1) KNOWLEDGE, SKILLS AND ATTITUDES: to what extent do you believe the following is a CORE outcome of international placements (that should be measured in a toolkit)?

2) ORGANISATIONAL OUTCOMES: to what extent do you believe the following is a CORE outcome of international placements (that should be measured in a toolkit)? 3) NEGATIVE OUTCOMES: To what extent do you believe the following is a CORE outcome of international placements (that should be measured in a toolkit)?

155

6.3.2. Rounds

After round two, 98 of the 156 statements were retained, this meant over 70% of the stakeholders agreed or strongly agreed these 98 statements were core outcomes. After re- considering their own vote in round two, the group median and anonymous comments regarding each statement 13 additional statements were retained in round three. Finally, after readdressing the above items for a second time an additional five statements met consensus and were retained in round four (see table 5). Of the items that met consensus 99 were positive and eight were negative. Positive outcomes were of educational benefits to the British health professionals and negative outcomes were drawbacks, costs or negative effects. Table 7 shows how the outcomes matched to the Jones et al., (13) framework, table 6 shows items that fell within more than one category. See appendix 3 and 4, for a full list of items and consensus levels.

Table 5: Number of statements with consensus at each round

Table 6: Examples of core outcomes that fell within more than one categories

Example Categories

Increased Awareness/Knowledge about clinical conditions and procedures rarely encountered in the UK

Clinical, Academic

Increased awareness of/knowledge about the importance of mutual learning and respect

Patient experience and Dignity, Communication and Team Work Ability to disseminate best practice globally Communication and Team Work,

Academic, Service Improvement and Policy

Ability to develop friendships Personal, Communication and Team Work Round Number of Statements with consensus (n=156) Consensus to include Consensus to exclude 2 98 97 1 3 13 10 3 4 5 1 4

Did not meet consensus

156

Table 7: Applying my results to the current knowledge: my core learning outcomes presented within the existing domains from Jones et al. (14)

Domain in Jones et al. (13) Number of COs within this domain

Examples

Clinical skills 12 Ability to use a broader range of clinical skills

Increased awareness of/knowledge about tropical diseases

Increased awareness of/knowledge about the cultural aspects of health

Management skills 16 Ability to be adaptable in leading

Ability to work within a system with unfamiliar power dynamics Ability to manage projects

Communication and teamwork 21 Understanding that words and behaviours can have different meanings Ability to co-operate

Ability to work as part of a team

Patient experience and dignity 19 Understanding own potential to empower people Increased respect for other cultures

Appreciation of free universal health Service/Policy development and

implementation

15 Increased awareness of/knowledge about the positive impact of clinical policies and governance

Appreciation of excellent human resource in the NHS

Academic skills 9 Ability to dissemination best practice globally

Improvement in teaching skills ability to build a global network Personal satisfaction and interest 16 Ability to develop friendships

Refreshment and reinvigoration Can-do attitude

157

6.3.3. Thematic results

The three charts below show how the results of the Delphi study relate to the key themes that came from the literature review: communication, leadership and cultural learning. Each shows the percentage of stakeholder consensus that was met for each component of the complex skill sets.

Figure 18: Percentage consensus for communication statements 91 86 84 83 76 76 72 70 0 0 10 20 30 40 50 60 70 80 90 100

WORDS AND BEHAVIOURS CAN HAVE… SPEED AND LANGUAGE COMPETENCY AFFECT… HOW CONTEXT AFFECTS COMMUNICATION

ABILITY TO OVERCOME COMMUNICATION… ABILITY TO ESTABLISH COMMUNICATION… ABILITY TO COMMUNICATE NON-VERBALLY

ABILITY TO VERBALISE KNOWLEDGE ABILITY TO ENGAGE SENIOR PEOPLE ABILITY TO LISTEN… Percentage Consensus St at e m e n t 100 100 93 91 90 88 83 0 0 10 20 30 40 50 60 70 80 90 100

CULTURAL DIFFERENCES AND SIMILARITIES CULTURAL ASPECTS OF HEALTH CULTURE IN PRACTICAL ASSESSMENT CULTURAL SENSITIVITY RESPECT FOR OTHER CULTURES GLOBAL ISSUES ISSUES OF EQUALITY AND DIVERSITY REINFORCED ETHNIC AND CULTURAL IDENTITY

Percentage Consensus St at e m e n t

158

Figure 20: Percentage consensus for leadership statements