Methods

4.2.1 Setting and Participants

Study 1 took place at the OU with a focus on undergraduate modules because they accounted for the largest number of students and subsequently had the highest proportion of students dropping out (Nguyen, Thorne, et al., 2018; Rienties & Toetenel, 2016b). These modules have a strategic position within the OU curriculum and need to be designed well as students do not necessarily have

64 the prerequisite qualifications required by other universities. This is because as its name suggests the OU is open to all. If students fail, then their future opportunities become limited and the mission of the OU is at risk.

Given the focus of the RQs on temporal characteristics of LDs, 56 modules were selected from the Activity Profile tool which mapped modules on a weekly basis (see section 4.2.2 below for a detailed description of the mapping process). After excluding 14 short and intensive training modules, be- cause they did not count toward academic credit, Study 1 used only 42 modules. The next step was to filter out five postgraduate modules because these modules consisted of a small number of stu- dents, whose background is not comparable to most OU students. Therefore, Study 1 was con- ducted on 37 undergraduate modules.

Using descriptive statistics for these 37 modules, it is clear from Table 15 below that there was approximately an equal distribution of 30 to 60 credit modules investigated. With respect to the levels of study, level 1 modules accounted for the largest percentage (70.3%), followed by level 2 modules (13.5%), level 3 and access modules (8.1%). The sample was distributed across all the four faculties at the OU, with the highest frequency in STEM (35.1%), followed by Arts and Social Sci- ences (24.3%), Education, Health, and Languages (24.3%), and Business and Law (16.2%).

Table 15: Descriptive statistics of 37 modules

Frequency Per cent Credits 30 17 45.9% 60 20 54.1% Level 0 3 8.1% 1 26 70.3% 2 5 13.5% 3 3 8.1% Faculty

Arts & Social Sciences 9 24.3%

Business & Law 6 16.2%

Education, Health, Languages 9 24.3%

STEM 13 35.1%

Note: Level 1, 2, 3 at the OU are equivalent to introductory, intermediate, and advanced courses. Level 0 represents access modules

4.2.2 Instruments

Data for Study 1 was collected from the Activity Profile tool (Toetenel et al., 2016a; Whitelock et al., 2016) which was designed to help teachers map different types of learning activity across a course or sequence of learning events (see section 3.2.2). The tool was developed based on the OULDI’s learning activity taxonomy which consists of seven types of learning activity: assimilative,

65 productive, assessment, communication, finding and handling information, interactive, and experi- ential. A detailed discussion about this taxonomy can be found in chapter 2 and the measurement can be found in chapter 3.

To ensure the quality of data, two approaches were taken by the researcher. Firstly, throughout the process, I maintained continuous discussions with four learning designers in the Institute of Educational Technology, who were responsible for the mapping process of these modules. I also joined several internal meetings of the LD team to understand the module mapping protocols. Sec- ondly, I carried out independent cross-checking with each selected module based on its online mod- ule guide available on the OU website (Figure 11).

Figure 11. Sampling process

4.2.3 Data analysis

To address RQ1.1, a combination of data visualisation, descriptive statistics, and correlational anal- ysis was used to explore the overall trends within the data. The data visualisation was completed using Tableau 10.1.6 and the descriptive statistics and correlational analysis were done using SPSS 23.

To answer RQ1.2, network analysis was used as this technique enables us to quantify and visualise the interactions and connections between the seven types of learning activity. A discussion about the background of network analysis can be found in chapter 3.

While the application of network analysis in education has primarily focused on modelling interac- tions between students, there has been very limited studies applying network analysis to model interactions between learning activities. To help readers understand the data analysis process, Ta- ble 16 showed an example of an LD mapping for 4 weeks. For example, in week 1, students were expected to spend 3.8 hours on readings, watching, listening activities and 0.8 hours on productive activities.

All weekly mapped

modules

•55 modules

Exclude short

training modules

•42 modules

Exclude postgradute

modules

•37 modules

Cross check with 4 learning designers

Cross check with OU online module guides

66 Table 16. Example of an LD mapping at a weekly level (unit=hours)

Week 1 Week 2 Week 3 Week 4 Assimilative 3.8 4.3 2.8 1.3 Information Communication Productive 0.8 3.9 2.6 1.4 Experiential 0.5 Interactive Assessment 1.8 3

This LD mapping was a weighted two-mode network as it consisted of different learning activity types (mode 1) across several weeks (mode 2) as illustrated in Figure 12 below. Since I am primarily interested in the relationships among learning activity types, the dataset was transformed into a one-mode network in line with Hora et al. (2013). In doing so, two assumptions were made.

Figure 12. A weighted two-mode network of module X across the first five weeks

Firstly, two learning activities (blue nodes) become connected if they were present in the same week (red nodes). For example, if teachers allocated 3.8 hours for assimilative (e.g., readings) and 0.8 hours for productive activities in week 1, then assimilative and productive activities become connected (Figure 13).

67 Figure 13. Transformation of a two-mode network into a one-mode network

However, simply visualising the connection between two activity types does not tell us much about the strength of the relationship. For example, module A with 5 hours of assimilative and 1 hour of productive activities will might look the same as module B with 1 hour of assimilative and 1 hour of productive. Since we captured how much time students were expected to spend on each LD each week, the weights of the two learning activities had directed towards identical weeks could also be measured. In this type of projected network, the weight of a tie from one LD to another was not necessarily equal to the weight of the reverse. For example, in Figure 13, if 3.8 hours were spent on assimilative activities and 0.8 were spent on assessment activities in the same week, then the weight from assimilative to assessment is recorded as 3.8 and the weight of the reverse is recorded as 0.8.

Second, the weight of each tie was discounted for the number of learning activity types in the same week (Newman, 2001). It can be argued that the tie between the two activity types becomes weaker when there are more activity types that are present in the same week. A simple analogy is the connection between two people is stronger there are fewer people in their group. This can be generalised as follows:

𝑤

= ∑

𝑤

𝑝

𝑁

− 1

𝑝

where wij is the weight between LD i and LD j, and Np is the number of learning activities in week p.

After transforming the dataset from two-mode to one-mode network, I used the Netdraw function of UCINET 6.627 (Borgatti et al., 2002), which is based on non-metric multidimensional scaling (Kruskal, 1964), to visualise the co-occurrences between each pair of learning activities across all weeks. The stress value was computed in order to determine the number of dimensions. Since all the stress values of two-dimensional scaling were far below 0.2, the graphs were visualised in two- dimensional space (Everton, 2012). The nodes represent different learning activity types. The tie

68 represents the co-occurrence of two learning activity types in the same week. The thickness of the line reflects the strength of the ties. In other words, the thicker the line, the higher the weights of the tie between two learning activity types.

In addition, descriptive network metrics were reported to support the reader’s interpretation:

• Network density: The percentage of existing ties out of all possible ties. The higher the den- sity, the more variety of combinations between activity types was used in an LD.

• Out-degree centrality: The frequency of an activity type was used with other types

• In-degree centrality: The frequency of other activity types was used with an activity type To address RQ1.2, which examines how teachers combine different types of learning activity in their LD, the first part of network analysis assumed that two learning activity types were “connected” if they were present in the same study week. For example, if week 1’s learning activities consist of assimilative activities (i.e. readings) and productive activities (i.e. open-ended questions), then as- similative and productive types are connected. The goal of RQ1.2 is to illustrate the diversity in how teachers mix and match different types of learning activity across modules. Due to limited space, I chose to report four exemplary modules from four different disciplines out of 37 modules to high- light different variations in LD.

While the first part of network analysis considers interactions between activity types at a weekly level, one could argue there are multiple learning tasks within a week. Thus, how teachers com- bined different activity types depends on the nature of each individual task. Therefore, the second part of network analysis was conducted at a learning task level. That means two activity types were connected only if they were used in the same learning task. For example, activity 1.1. consists of readings and finding information, then assimilative type and finding information type are con- nected. This fine-grained network analysis was carried out on 268 individual learning tasks on a level 1 Social Sciences module because this module has been mapped at a task level.