Think-aloud analysis

The six participants that were involved in Experiment 3 used the think-aloud technique to focus on the design of the materials compared to using a traditional textbook to learn about a topic. This method of employing the think-aloud technique confirms Pressley and Afflerbach’s (1995) claim that participants who employ this technique with what are considered to be simple tasks have difficulty verbalising the cognitive processes as opposed to verbalising issues related to the learning task itself. When the ‘Model of human memory and information processing’ is taught at university, students typically receive an entire lecture on the model. In textbooks, it typically takes a few pages to explain the concept. The applications in this study were designed to simplify the process by presenting the information within the model (removing split-attention) and learning the information in a similar way to a worked- example. Participant G3-1 stated that the application was “very simple, a basic program” – this comment confirmed that the aim behind developing the application (i.e. making a typically complex procedural process easy to learn) was achieved. With each participant only being exposed to one instructional method, the results from this experiment are designed to aid developers in best utilising each style of interaction in future applications.

5.4.2.1 Before interacting with the Flash web-based applications

Initially, all of the participants were presented with the same model of the application. Once the participants began to click through the application, the information was presented in different ways. The following comments are based on initial impressions about the application.

Participant G1-1 noted that there was initially an input arrow; this demonstrated that the participant realised that the model consisted of a process. Participants G2-1, G2- 2, G3-1 and G3-2 took this a step further by reading all of the content on the screen (the basic model of ‘Model of human memory and information processing’). These participants identified that there was an overall model that was presented by the application.

5.4.2.2 G1: Integrated layering with no previous information displayed

As stated before, this application removed the information a participant had just read when a participant moved to the next piece of information. As in Experiments 1 and 2, the participants who evaluated this model did not achieve marks that were as high as G2 and G3 in the post-test questions. Therefore it was assumed that they did not achieve the same level of learning outcomes as the participants that were involved in G2 or G3.

The first participant examining this model verbalised that he had to rely on just the overarching model for understanding how the ‘Model of human memory and information processing’ fit together after reviewing the 12 stages of the model. “Firstly my understanding is the input comes from the sensory register and then processing in working memory which is then encoded into long-term memory but long-term memory access retrieval is from working memory” Participant G1-1. This insight is from the overall model, as participants using the G1 application are only able to see the overarching model and the last piece of information at the end of the application. Participant G1-2 did not complete a final review of the model, but instead focused on how the content was displayed on the screen.

It was expected that the participants reviewing the G1 application would make comments about the information not being able to be re-read as they moved throughout the application; however, this was not the case. Further research could be conducted to examine whether removing the previous steps is an issue for users of web-based applications. Even though it was not verbally identified as a problem, this is the only difference in the applications and there were significant differences in learning outcomes when comparing G1 to G2 and G3.

5.4.2.3 G2: Integrated layering with previous information displayed

This application built the information as the participant moved through the 12 pieces of information. The participants that used G2 achieved the highest marks of all three applications in the three experiments.

Participant G2-1 identified during the tutorial that it seemed important that the information was available for re-reading, “so I know that it is helpful”. This not only

tutorial that was developed for Experiment 2 helped the participant make best use of the layered integrated application. This tutorial was not available to participants in Experiment 1. At the completion of the application’s usage, the participant reiterated that he found it useful for the information to be always displayed - “yes I think that it is good to keep the previous steps up” - and how he re-read some of the information. This demonstrates that the participant was able to obtain the context by keeping the information available and was able to use the steps as interactive elements.

Participant G2-2 was focused on the design of the learning materials, stating that the concept of putting the information in the model aided in their learning. This concept is supported by the goal of the application: removing split-attention from the learning process.

5.4.2.4 Integrated layering with the current step highlighted

This application built the information as the participant moved through the 12 pieces of information in a similar way to the G2 application, however the previous step was made grey so that the current step was highlighted. The participants that completed G3 in Experiment 2 achieved marks that were statistically higher than G1, however not as high as G2.

Participant G3-1 identified that they did not fully understand the purpose of the tutorial or how they were meant to progress (i.e. click the next button) through the tutorial. This user considered that they had a high level of computer usage. A potential area of further investigation could be the use of tutorials compared with a video of how to use the application.

Participant G3-1 noted that the application was easier to use than a textbook as it provided less information.

Participant G3-2 used their own example of trying to memorise a telephone number when working through the application. This demonstrated that they were attempting to use an example to relate the content back to their own experiences, and that the participant was imagining the information to enhance their learning. Leahy and Sweller (2004) have previously identified this as a method to increase the use of germane load within a learners’ WM. Finally, Participant G3-2 noted “this whole screen here was actually very interesting, like a kind of way presented as a story”.

This identified that the presentation of the application was able to show the participant the entire model, and the participant was able to engage with all the interacting elements.

5.4.2.5 Common issues across all three web-based applications

During the tutorial, participants were taught that the applications were designed to be clicked through sequentially. However, once participants started to use the applications this instruction was forgotten when animations occurred in the applications.

From the participants’ comments, the applications seemed to exert a level of ECL when the arrows appeared between each of the different memory structures (animations). This had a tendency to confuse participants and removed their focus from learning the ‘Model of human memory and information processing’. The applications were designed to focus the learner when information in the box that they were reading indicated that at that stage in the process the information being learnt transferred to the next memory structure, i.e. from SM to WM. The comments such as “I thought it was actually a signal to process box 7” (from Participant G1-1) and “it is very confusing at the start when after the second click an arrow wants you to go across to working memory” (from Participant G3-1) were common among all participants.

These animations were designed to highlight important information for the participants, as presented below. Some participants (e.g. G2-2) identified that they liked the animations while other participants (e.g. G1-1) identified that the animated arrow indicating lost information did not add anything to the application. However, this is an important part of the model. Participant G3-2 quickly realised their error and identified that “I clicked on that box, I was directed to this box over there, I thought that must be what the next step is, but I quickly realised the arrow was to simply simulate the concept”. This comment revealed the potential that an animation has to control participants’ direction in an application.

What is important when designing applications around a model is how the overarching model is presented on the screen. As participants worked through the application, they typically read the static material (i.e. headings for each memory

structure) as they worked through the learning materials. Some of the participants (e.g. G1-2 and G2-2) suggested that the headings of the learning task and the headings for each memory structure should be bigger. Participant G2-2 also noted that different colours should have been used to make the different memory structures within the model clearer.

5.5 Conclusion

This chapter has presented the results of the three experiments that were used in this study to evaluate the three different instructional applications designed to present the ‘Model of human memory and information processing’ through the use of layered integrated instruction. Generally, the main concern for instructional designers is to develop the most appropriate method for the organisation and presentation of information; this needs to be done taking into consideration a learner’s ability to process information.

This study has emphasised the effectiveness of layered integrated instructional designs in creating learning outcomes for learners. The following chapter presents a general discussion of the results and its impact on theory and practice.