General set-up of the experiments

All three experiments have a similar set-up and look for the same measurements. The research design of the three experiments is a combination of exploratory and experimental research. The participants study written materials on cloud identification, play a game (voluntarily or mandatorily), and then take a test. All participants are free to choose how much time they spend on studying the written materials.

Participants who play the game voluntarily are free to choose how long they play, including not playing at all. Participants who play the game mandatorily have to play for at least ten minutes. The procedure is described in more detail in Subsection 4.2.3, and the materials are described in Subsection 4.2.4.

The second and third experiment build on the previous experiments, using the same procedure and materials but with some adaptations. The adaptations made for the ex- periments are described in Sections 4.3 and 4.4.

In all three experiments, we differentiate between the participants based on two inde- pendent variables, viz. (1) participation, and (2) gameplay.

1. Participation. Participants who have volunteered to participate in the experiment are voluntary participants (VP). In contrast, the participants who partake in the experiments as part of a school assignment are mandatory participants (MP).

2. Gameplay. Participants who are free to choose whether they play the game and how long they play the game, are voluntary players (VG). In contrast, the participants for whom a minimum of ten minutes of gameplay is enforced, are mandatory players (MG).

Experiment 1

The first experiment (Figure 4.1) had an informal, non-educational setting. It took place in July and August 2015. A total of 19 voluntary participants (VP), with no link to aviation, were randomly assigned to either voluntary gameplay (VG) or mandatory gameplay (MG). Moreover, a reward was offered to the participants. The experiment yielded interesting outcomes that led to a second experiment. Experiment participation, even when assigned to the mandatory gameplay (MG) condition, has a much stronger voluntary character than GBL usually will have. Therefore, the second experiment was set up as part of a course to create some level of mandatory participation (MP). We will discuss the results of Experiment 1 in Section 4.2.

Experiment 2

The second experiment (Figure 4.2) was carried out from March to May 2016, with a total of 74 participants. A total of 71 participants had no link to aviation. The experimental design and materials were adapted to include mandatory participants (MP) in addition to the voluntary participants (VP), and to introduce a no-gameplay (NG) condition in addition to voluntary (VG) and mandatory (MG) gameplay. Some of the participants received an incentive or reward for participating. Again, the outcomes of the experiment

4 Participants n = 19 Informally recruited, no perceived utility, rewards offered Mandatory Participation (MP) Mandatory Gameplay (MPMG) Voluntary Gameplay (MPVG) Voluntary Participation (VP) n = 19 Mandatory Gameplay (VPMG) n = 9 Voluntary Gameplay (VPVG) n = 10

Figure 4.1: Structure and number of participants of Experiment 1

were interesting but inconclusive. Hence, a third experiment was conducted. We aimed to create an even stronger sense of mandatory participation (MP). We will discuss the results of Experiment 2 in Section 4.3.

Participants n = 74 Recruited in higher education,

no perceived utility, rewards offered partially

Mandatory Participation (MP) n = 28 No Gameplay (MPNG) n = 11 Mandatory Gameplay (MPMG) n = 6 Voluntary Gameplay (MPVG) n = 11 Voluntary Participation (VP) n = 46 No Gameplay (VPNG) n = 13 Mandatory Gameplay (VPMG) n = 15 Voluntary Gameplay (VPVG) n = 18

Figure 4.2: Structure and number of participants of Experiment 2

Experiment 3

The third experiment (Figure 4.3) took place between February and August 2017, with a total of 83 participants. This experiment had a formal setting with a stronger distinction between voluntary (VP) and mandatory (MP) participation. No incentives or rewards were offered to eliminate possible confounding variables. All participants were (aspiring) pilots. Therefore, the topic of the experiment had a stronger relevance and utility for them.

The third experiment confirmed, to some extent, the interesting outcomes of the previous experiments. We will discuss the results of Experiment 3 in Section 4.4.

4 Participants n = 19 Informally recruited, no perceived utility, rewards offered Mandatory Participation (MP) Mandatory Gameplay (MPMG) Voluntary Gameplay (MPVG) Voluntary Participation (VP) n = 19 Mandatory Gameplay (VPMG) n = 9 Voluntary Gameplay (VPVG) n = 10

Figure 4.1: Structure and number of participants of Experiment 1

were interesting but inconclusive. Hence, a third experiment was conducted. We aimed to create an even stronger sense of mandatory participation (MP). We will discuss the results of Experiment 2 in Section 4.3.

Participants n = 74 Recruited in higher education,

no perceived utility, rewards offered partially

Mandatory Participation (MP) n = 28 No Gameplay (MPNG) n = 11 Mandatory Gameplay (MPMG) n = 6 Voluntary Gameplay (MPVG) n = 11 Voluntary Participation (VP) n = 46 No Gameplay (VPNG) n = 13 Mandatory Gameplay (VPMG) n = 15 Voluntary Gameplay (VPVG) n = 18

Figure 4.2: Structure and number of participants of Experiment 2

Experiment 3

The third experiment (Figure 4.3) took place between February and August 2017, with a total of 83 participants. This experiment had a formal setting with a stronger distinction between voluntary (VP) and mandatory (MP) participation. No incentives or rewards were offered to eliminate possible confounding variables. All participants were (aspiring) pilots. Therefore, the topic of the experiment had a stronger relevance and utility for them.

The third experiment confirmed, to some extent, the interesting outcomes of the previous experiments. We will discuss the results of Experiment 3 in Section 4.4.

Participants n = 83 Recruited in flight schools,

perceived utility, no rewards offered Mandatory Participation (MP) n = 53 Mandatory Gameplay (MPMG) n = 26 Voluntary Gameplay (MPVG) n = 27 Voluntary Participation (VP) n = 30 Mandatory Gameplay (VPMG) n = 13 Voluntary Gameplay (VPVG) n = 17

Figure 4.3: Structure and number of participants of Experiment 3 Overall discussion of the three experiments

In Section 4.5, we will give an extendsive overview of the results of all three experiments.