3. CHAPTER 03: METHODOLOGY
3.9. DATA COLLECTION PROCEDURE
3.9.1. Stage 01: Participant Information and Consent
Participants were given a thorough explanation of the research project and the researcher talked them through the participant information form (Appendix D :) and asked if there were any questions. The researcher explained that consent could be withdrawn at any time and that participants could opt not to submit the data collection forms at the end of the collection activity if they did not wish to be included in the study.
3.9.2. Stage 02: App installation
The participants in the experimental group were provided with a link to the online web app described on page 138. The link was provided in advance to allow them to download and open the application in readiness for the learning task.
3.9.3. Stage 03: Pre-Test
The pre-test, NASA TLX and post-test were handed out as paper documents to all of the participants (these can be found in Appendix B :). The pre-test was conducted by screening the skull-base image over the conference projection system and asking the participants to write down the names of as many of the arrowed structures as they could identify in a ten-minute timed session. The rationale for projecting the image was to ensure that none of the participants could access the pre-test content before the task started as this was a timed exercise. The rationale for controlling the time variable was to ensure that all participants in future data collection sessions would have an equal amount of complete the pre and post-tests. During the first data collection session, it was noted that one participant had returned to the pre-test answer-sheet after the learning activity and had attempted to fill in the answers that they had failed to complete during the pre- test. This paper was voided after collection and not used in the data analysis. Following this incident, participants in all future data collection sessions were asked to draw a line through any of the boxes on the pre-test answer sheet that they were unable to
complete. This strategy prevented participants from filling the answers retrospectively following the learning activity.
3.9.4. Stage 04: Learning Activity
Control Group
The control group were provided with a labelled photograph of the anatomy to be learnt
(see Figure 3-3) and asked to study it for 10 minutes and to learn (memorise) as many of the labelled anatomical structures as possible. Another potential source of data
invalidation was spotted at a UK data-collection session wherein it was identified that one participant was writing down the names of the anatomical areas during the learning activity. Having these cues to refer to in the post-test would have invalidated the data, as the answers would not reflect learning. This paper was, therefore, voided and the data not used in the analysis. For all of the following data collection sessions, it was explained to the participants that no writing was permitted during the learning activity.
Experimental Group
The experimental group were asked to open their web apps (see Figure 3-2) and were provided with a pass-code to open the learning activity. They were then given ten
minutes to learn the labelled anatomical structures. The code was required to ensure that the experimental group did not open their learning activity before the pre-test had been completed as this would have provided the answers. The experimental group were asked to remain in the same environment as the control group to ensure that there were no confounding variables such as distraction from other sources.
The interactive activity featured the same photograph of the skull-base but allowed the participants to tap on the various foramina. By tapping the screen, auditory and visual information was presented to the learner identifying the name of the foramen in question and the structures passing through. In
Figure 3-2 the screenshot shows the appearance presented when the participant taps on the greater palatine foramen. The coloured outline of the foramen is highlighted (from being semi-opacified to fully opacified) simultaneously with a short, spoken audio description identifying the structure in question. Following the audio description, the software presented a text label next to the foramen (Figure 3-3). The same audio-visual highlighting and labelling could be activated by tapping on each of the foramina shown in
the photograph. The interactivity of the screen also permitted the users to tap the corresponding foramina on the opposite side of the screen (the skull base being symmetrical left to right) and cause the same information to appear. The photograph embedded in the mobile app was presented at a higher resolution than the typical mobile device screen size and was, therefore, zoom-able without loss of detail (by using the default pinching gesture on the screen). There were a set of coloured buttons located down the left of the screen that could be clicked to highlight the foramina more easily than clicking an un-zoomed diagram. These features were intended to aid participants using small screen sizes such as smartphones.
The principles of Cognitive Load Theory (CLT) were applied to the material design to reduce confounding additional extraneous cognitive load imposed on the participants due to the split-attention principle (Nielsen, 1994; Mayer, 2009) or the redundancy principle (Mayer, 2009; Sweller & van Merriënboer, 1998). To avoid cognitive load imposed by the split-attention effect, it was important that the learning tasks should not require the participant to mentally integrate multiple information sources. This spatial contiguity was achieved by ensuring that the diagram and labels for the control-group task were not spatially separate and that the text labels could be viewed simultaneously and in close proximity to the corresponding foramina shown on the photograph (Mayer, 2009).
To avoid extraneous cognitive load imposed by the redundancy principle, it was necessary to ensure that images, text and spoken narration/audio were not presented
simultaneously (Paas, et al., 2003; Liu et al., 2012). For this reason, the audio cue was presented before the appearance of the text labels in the interactive presentation provided to the experimental group.
The learning activity session was carefully invigilated to ensure that none of the
participants wrote down any information that could have been used in the post-test. The control group activity was timed, and the participants were asked to stop the activity after 10 minutes had elapsed. The experimental group activity was automatically timed, and was closed by the app after 10 minutes. It could be hypothesised that the presence of the timer on the screen of the device might have increased extraneous cognitive load if the experimental group had been distracted by it during the activity. This variable was
controlled by presenting the same timer as a large projection on the screen in the data- collection room for the control group to refer to during the task.
3.9.5. Stage 05: NASA TLX Questionnaire
The task load relating to the learning activity was assessed using the NASA Task Load Index. For the NASA TLX, the instructions were screened to the participants over the main conference projection system and the researcher went through the instructions and descriptors before asking the participants to indicate their responses on the paper data collection forms provided. Both sections of the form were used to assess the load placed on the participants by the six dimensions of the tool and also the pair-wise choices to enable weighting of the results. To minimise procedural bias, this activity was not timed, and enough opportunity was given for all participants to complete their responses. One issue arose during the NASA TLX data collection that required papers to be declared void. Some participants did not fully complete both sections of the NASA TLX, usually by failing to indicate one or more of the pair-wise comparisons. To avoid this happening in future data collection sessions, the researcher asked all participants to double-check their sheets at the end of each session to ensure that they had completed all of the sections correctly.
3.9.6. Stage 06: Post Test
The final data collection activity was the post-test. The test was administered under the same conditions as the pre-test. The reason for using a post-test was to allow the pre-test score to be subtracted from the post-test score to give a measurement that could be triangulated against the reported task load to see if there was a correlation to support the assumption that post-testing is an indirect measure of cognitive load. The reason for presenting this test after the NASA TLX rather than directly after the learning activity was to ensure that any knowledge gained during had been committed to long term memory. The activity was invigilated to ensure that participants did not seek to gain an advantage by filling in the pre-test section retrospectively.
After the post-test, there was a text-field on the data collection form to allow the
participants to provide an email address (not compulsory) and demographic information relating to their age, gender and the type of device used.
To ensure anonymity participants were asked to fold their sheets and place them in a box when leaving the room. It was explained that if anyone wished to withdraw from the study at this point that they could discard or keep their data-collection sheets rather than submit them to the collection box.
This concluded the data collection activity. The above process was conducted six times over a twelve-month period with groups of approximately 20 participants in each instance.