Development and Administration of Learning Anatomy Survey and Interview

and Interview Questions

The learning anatomy questionnaire was developed from a similar tool that was pilot tested on a previous cohort, from which findings were presented at conferences.

The qualitative approach to Phase Two of the research involved conducting an interview session with participants to explore their experiences with learning, forgetting and retention of anatomy knowledge. The interview questions utilised for this research were developed as part of an existing research project and modified for the purposes of this study.

The purpose of the tool was to explore the research questions and to gain a better understanding of the data obtained through the quantitative analysis. The interview aimed to focus on topics relating to how students learned anatomy; what approaches and teaching methods they preferred for mastering anatomical knowledge (particularly in Year A), how much anatomy they felt they had retained, how much anatomy they had used in the clinical setting and how important they perceived anatomy to be in the clinical setting.

Using Seidman (2013) and King and Horrocks’ (2010) guide to interviewing and qualitative research, a set of questions was formulated for use during the interviews, as listed in Appendix G.

5.7

Data Collection

5.7.1 Phase One

Once consent had been obtained, participants were given an approximate

timeframe and advised that, one week prior to commencement of the data collection for Phase One, they would receive an email containing a link to the secure online anatomy assessment site, along with their user ID and password. The user ID for accessing the online tool was the participant’s student ID. Each student was also assigned a randomly generated password. Participants were instructed that the link could only be accessed on the specified day for a period of 24 hours, and if they tried to access it before then, the site would display ‘This page is not available’.

The clinical-year students (Years B–D) had upcoming summative assessments in early November, and the Year A students were due to complete their final week of classes in early November. Therefore, two separate dates were nominated for the data collection for this quantitative phase. The data collection for participants in Years B–D took place in the last week of October, whereas the data collection for participants in Year A took place in the first week of November.

On the date prior to that specified for Phase One, participants were emailed a reminder advising them that the site would be open in less than 24 hours. A similar email was sent on the ‘go live’ date, with participants able to access the online site by entering their user ID and password. Once logged in, participants were introduced to the research tasks and the research team via a welcome message. The purpose of the study was highlighted once again, and participants were informed about the two parts

associated with this phase. Thereafter, participants commenced the anatomy-learning questionnaire. Upon completion, participants were directed to part two—the anatomy assessment task. An example screen displayed the style in which the questions would

appear, and the location of the countdown timer enabled students to visualise the appearance of the site, the placement of the questions and their options, and how to monitor their time for each question. Subsequent to this, students began the online assessment.

During the ‘go live’ dates, the student researcher and the site developer monitored students’ participation on the site. Those who had not yet logged on were sent a further email reminder to do so before the assessment task and site closed.

As with any online system, there were implementation issues. Participants would be unable to complete the task if they clicked on the ‘back’ button on their web browser; therefore, the research team implemented rules in case a student encountered this issue. It was decided that if the student had completed fewer than ten questions, they would be allowed to re-enter the site and complete the task. As was expected, four participants were accidently locked out of the site and were unable to progress further on the assessment task. The participants immediately notified the research team and, because they had not advanced very far on the task, their access to the site was reinstated and they were able to log on and complete the assessment. When analysing the data for these students, no changes were noted in their responses from their first attempt to answer the questions.

5.7.2 Phase Two

In a sequential explanatory mixed-methods study, the key purpose of the quantitative data collection, in addition to answering part of the research question, is to guide the development and selection of participants for the second follow-up phase, which involves the collection of qualitative data. Participants selected for Phase Two were emailed an invitation to meet for a face-to-face or phone interview. Ultimately, 13 interviews were conducted with participants from the four cohorts: three from Year A,

four from Year B, and three each from Years C and D. During the interviews,

participants were advised that all information would remain confidential. All interviews were recorded and later transcribed verbatim. Each interview lasted approximately 45 minutes to one hour. The shortest interview was 32 minutes and the longest was two hours.

5.8

Data Analysis

5.8.1 Phase One

All Phase One data (i.e., the learning anatomy questionnaire and the anatomy assessment scores) were downloaded as two separate Excel files by the site developer. These files contained raw data, so a clean-up of the data was carried out using SPSS Statistics version 22. To collate the responses from both the learning anatomy questionnaire and the anatomy assessment, the information from both files was

transferred and merged into one SPSS document. This ensured that subsequent analysis could be carried out using a number of key variables, and that comparisons could be inferred between the variables. The main areas of focus within the analysis included the following:

 examination of original data to determine extreme outliers and the subsequent exclusion of participants’ data

 statistical analysis of anatomy questions, including the calculation of reliability, SD and SEM

 overall performance of participants on the anatomy test and comparisons between the four cohorts

 regional performance of participants on the anatomy test and comparisons between the four cohorts

 effect of the three different types of questions (taxonomies) on participants’ performance on the anatomy test

 comparison of participants’ original Year A VIA scores and their performance on the anatomy test

 comparison of Year A participants’ performance on the anatomy test and the effect this had on their subsequent summative assessment (VIA)

 comparison of participants’ prior educational background and performance on the anatomy test

 teaching resources mostly used by participants in learning anatomy

 best approaches used by participants in learning anatomy and their relationship to performance on the anatomy test

 participants’ perceptions of anatomy.

The statistical tests utilised in the analysis of data involved a range of parametric tests such as ANOVA and paired t-tests where the data met the assumptions for normality (as assessed by skewness, kurtosis, Shapiro-Wilk, Q-Q plot and boxplot). This is followed by post-hoc tests for pairwise comparisons (Tukey-Kramer) and Pearson correlations for statistical significance. Where data did not meet normal assumptions, the analyses were undertaken using a series of non-parametric tests involving the Chi-square test of independence, Spearman’s rank correlation, Mann-Whitney test and Kruskal-Wallis H test followed by appropriate post-hoc tests for pairwise comparisons (Dunn’s procedure using Bonferroni correction). The results of the above analysis are presented in Chapter 6.

5.8.2 Phase Two

The qualitative data were analysed using thematic analysis (King & Horrocks, 2010) to determine which aspects provided insights into the quantitative data. Thematic

analysis is the process of identifying, analysing, interpreting and reporting patterns (themes) within qualitative data (Boyatzis, 1998; Braun & Clarke, 2006). This method allows for flexibility because it can be applied to a variety of theoretical frameworks and shaped or constructed in a way that is useful and that suits sequential designs (Braun & Clarke, 2006). Four decisions had to be made in relation to approaching the data to determine the form that the thematic analysis could take (Braun & Clarke, 2006).

The first criterion necessary to shape the analysis was to determine the content from the data set that could be categorised as a theme. A theme captures data that are essential in answering the research question. It contains prevalent and meaningful patterns of responses that can be found within a particular data item or across the whole data set (Trahan & Stewart, 2013). A data set refers to all individual data items that are collected within a data corpus (i.e., all data collected in a research project, such as quantitative and qualitative data) and that are important and relevant to the topic at hand. Therefore, the data set for Phase Two referred to all 13 interviews that were recorded, and the data item referred to each individual interview recording and transcript (Braun & Clarke, 2006). It is unclear how prevalent a concept must be to be classified as a theme. In the recent body of literature on thematic analysis, a void still exists around establishing universally accepted standards for recognising the relevance of a pattern. Instead, researchers are advised to use their own judgement in this process, and to pay particular attention to issues that are

relevant to the research question and/or issues that help explain the quantitative results and, in both cases, to ensure that a consistent style is applied to the entire process (Braun & Clarke, 2006; Trahan & Stewart, 2013). Using this approach, the data set was explored for repeated patterns of meaning, and a set of themes was identified and established. These themes are discussed in Chapter 7.

The second criterion involved the description of the data and whether it would consist of detailed accounts of the entire data set or only certain aspects of it. As Trahan and Stewart (2013) state, “In mixed methods research, the scope of thematic analysis should be guided primarily by the research model” (p. 66). Given the nature of the sequential explanatory model, wherein qualitative data were used to explore and explain the quantitative results, the data were examined for themes that related to the research questions and for themes that helped elaborate the specific findings of the quantitative data. That is, the description of the data as documented in the subsequent chapter does not contain a detailed account of each participant’s journey; rather, it highlights specific issues as they relate to the study.

The third criterion involved determining whether the themes identified should be determined via an inductive or deductive approach. An inductive or bottom–up

approach ensures that the themes identified are linked directly to the data; therefore, they may have little significance to the research questions. In contrast, a deductive or theoretical approach is explicitly driven by the research questions and the researcher’s interest in the area, and it often calls for engagement with the literature prior to thematic analysis (Braun & Clarke, 2006). Once again, the research model for the study assisted in this decision; accordingly, the sequential explanatory design was best served by a deductive approach to thematic analysis. This is because “the quantitative antecedent can be used to identify the most salient concepts and test relationships between them. The subsequent qualitative study can then be used to identify these concepts in the qualitative data and elaborate on their interactions” (Trahan & Stewart, 2013, p. 67).

The final decision to be addressed involved determining the ‘level’ of thematic analysis (i.e., sematic or latent analysis). A semantic approach involves identifying themes within the explicit and surface-level meanings of the data. These themes can

then be organised, described or summarised and interpreted to generate an explanation of a particular concept or pattern and its broader implications. However, a latent approach involves looking beyond the semantics to examine, analyse and theorise the themes identified within the data (Braun & Clarke, 2006; Trahan & Stewart, 2013). For this study, a semantic approach was used to generate themes across the entire data set.

Following the decisions made on the above four criteria, a six-step procedure to thematic analysis was followed, as outlined by Braun and Clarke (2006):

1. Familiarisation with the data: This involved engaging in a process of active reading and re-reading of the data to allow repeated patterns of meaning to be identified. The process began at the outset of data collection because the primary researcher was involved in conducting interviews with the participants. All 13 interviews were transcribed verbatim and then printed and compiled into one data set. During the reading phase, notes and coding ideas were listed against the transcripts for future use and analysis.

2. Generation of preliminary codes: Once familiar with the data, a set of initial codes was listed. These codes (semantic in nature) alert the researcher to data that are of interest to the study. As Boyatzis (1998) describes, codes offer “the most basic segment or element of raw data or information that can be assessed in a meaningful way regarding the phenomenon” (p. 63). Each data item (i.e., interview transcript) was read, and code words were generated manually using a theory-driven approach—that is, considering the research questions and the

quantitative results. The key issues identified within each data item were then reviewed, and further codes for the entire data set were developed.

Data extracts (individual chunks of information from a data item) were coded from each data item, and all data extracts for a particular code across the data set were collated. Some data extracts were coded more than once.

3. Creation of themes: Following a review of all codes generated in the previous step, a search was conducted for repeated patterns or themes. This involved analysing the codes from a broader perspective to identify general patterns (themes) and subthemes within those patterns across the data set. It also involved organising the data extracts under such themes. 4. Revision of themes: During this phase, the reliability and accuracy of the

themes identified in the previous step were examined (Trahan & Stewart, 2013) to ensure that the apparent patterns could indeed be classified as themes. Two criteria (internal homogeneity and external heterogeneity) had to be verified to ensure that the patterns constituted themes (Patton, 1990). Internal homogeneity refers to data within a theme that coheres in a meaningful way, whereas external heterogeneity refers to clear and identifiable differences that exist between the different themes. To assess these criteria, the data were analysed at two levels, as recommended by Braun and Clarke (2006). The first involved re-reading all of the data extracts for each theme to determine whether a meaningful pattern existed between them. If this was the case, then internal homogeneity was maintained. However, if the data did not fit into the general pattern of the theme, the data were either reassigned to another existing theme, a new theme was created to accommodate the data, the existing theme itself was reworked or the data were discarded from the analysis. The

final step in this stage of the analysis was the creation of a thematic map to capture the contours of the coded data. The second level of analysis involved re-examining (i.e., re-reading) the entire data set to ensure the validity of each theme in relation to the data, and to code any additional data that might have been missed the first time around. As Trahan and Stewart (2013) write, “the themes are considered heterogeneous if, without intersecting, they precisely reflect the content of the data without missing any important concepts” (p. 70).

5. Defining of themes: This stage involved further refinement of the themes that would later be presented for analysis by creating definitions that captured the essence of each meaning. The purpose of this was to create a detailed analysis of each theme to build a broader picture of its

significance to the research questions, and to ensure that there was no overlap with other themes. Braun and Clarke (2006) suggest that if the analysis has been undertaken correctly, it should be possible for the researcher to describe the content of each theme and its implications in only a few sentences.

6. Write-up of analysis: This phase, which is presented in Chapter 7 conveys the results of the analysis as clearly and concisely as possible. Evidence of the story obtained through the data is organised in a logical and non-repetitive way. Data extracts and vivid examples are used to illustrate the essence of the story being told. The chapter not only describes the qualitative results, but also provides an analytic narrative for the data to demonstrate how the results answer the research questions (Braun & Clarke, 2006).

In order to ensure triangulation data obtained through thematic analysis

(Creswell & Plano Clark, 2011), all researchers in the study were consulted in the data analysis of the interview transcripts and agreed on the themes identified. In employing these stages of thematic analysis—namely, descriptive coding, which involves

highlighting relevant material and assigning descriptive codes; integrative coding, whereby one groups and makes sense of the descriptive codes and their significance in relation to the research question; and overarching themes, which identify key themes across the entire data set (King & Horrocks, 2010)—a relationship was found between the main themes, which provided insights into the importance of anatomy teaching, retention and knowledge in clinical performance. The results of the quantitative and qualitative strands were explored together to determine the outcomes of the research questions.