4. Methods
4.7. Generating preliminary findings
The second purpose of the pilot was to generate preliminary findings. Whilst the sampling and design of the pilot were developed in order to conduct the principal component
analysis, I was able to gather data about how the tool works in practice by performing a secondary analysis on the results.
107
4.7.1. Between-group comparisons
On the large pilot data set, I planned a between-group statistical analysis (Mann Whitney U tests) of the different groups that completed the tool. Hence, I was able to compare the scores of those with and without international experience to understand whether the tool is sensitive to differences between-groups. I was also able to compare across professions, age groups, career stage etc. to understand how individuals perform generally on each of the items.
4.7.2. Within-participant comparisons
Within the pilot I captured pre-departure data for professionals that were due to undertake an international placement in the coming months. Although it was only a small number of participants, it captured a baseline measure of levels of the skills, knowledge and attitudes. I was then able to recapture this data one year later and measure any change within the individuals after their international experience. I conducted Wilcoxon signed-ranks tests to statistically measure the difference in latent traits before and after the international
placement.
4.7.3. Interaction between variables and PPD outcomes
Output 2, a list of potential variables that affect learning on international placements was generated earlier in the research process as a result of the systematic review and meta- synthesis. I created a list of all of the factors that were reported in the literature to potentially affect learning or PPD. I captured this data within the pilot and alongside the learning outcomes, by asking demographic questions, and questions about the contextual and environment components of international placements. Therefore I was able to look for patterns and trends that may give an indication about how certain contextual components of an international environment have a relationship with learning.
This secondary analysis was conducted on data of participants with past international experience as they were the only group able to provide the contextual data. I performed statistical analysis (Kruskal Wallis H tests) comparing those who experienced low, medium or high levels of a variable on their placement, with their scores on the PPD domains. This allowed me to look for patterns without attributing effect. For example, I
108
was able to assess whether those who reported high levels of interaction with patients, presented higher scores on the ‘adapting communication’ domain than those who saw less patients.
4.8. Summary
In this chapter I presented the numerous outputs I intended to generate and the various methods that I have used to generate each output. The research process was progressive and each output was a product of the previous output (readdress figure 11 for a visual depiction of the whole process). In summary, the first method used was a systematic review, this allowed me to extract every potential PPD outcome, negative outcome and variable that was reported in peer reviewed literature in regards to health professional international learning. The results of this were then meta-synthesised to reduce the number of potential outcomes and remove profession or individual specificity. The third method used was a Delphi study, I took the outcomes from the meta-synthesis and asked
stakeholders to make judgements about their applicability to the research question. This resulted in the development of a core outcomes set (COS), an agreed upon set of PPD outcomes of learning on international placements. The fourth method was a pilot study, whereby I converted these outcomes into self-report items and presented them to health professionals to understand which of the items have the best psychometric properties. On the results of which I used Principal Component Analysis, a statistical technique that reduces large sets of data into its principal components (and items that provide the most psychometric information). Finally, I conducted secondary analysis of the results of the pilot to understand how the tool works, the scores that people get, the relationships between different groups, between individuals longitudinally and patterns and trends between variables (contextual factors) and PPD outcomes.
In the next chapters, I describe the background to each method in regards to academic literature, the precise method used in each of the components of the study and the results and discussion of each. Each of the next five chapters describe each progressive study. The first will describe the development of the core outcome set, the next the Delphi, then the piloting of the tool and principle component analysis and multivariate item response theory. Chapter 8 will discuss how the tool can be used to compare groups with and without international experience and differences in scores post placements. Chapter 9 will describe the secondary analysis of the pilot data, looking for relationships between
109
variables and outcomes. To see the visual summary of how the aims, methods and outputs interact as a whole entity see figure 11.
110