Design of Study

Instructional materials on expected values and the principles for calculations

were derived from Winston (1994). Whilst the expected values’ options were referred to

replaced with the word ‘game’ to minimise confusion as the constructive tasks referred

to real-life applications such as insurance. Expected values tasks were chosen and

modified from Winston (1994) to fit the taxonomy of Galbraith and Haines (2000a). For

the mechanical and interpretive tasks, there were three games each that students had to

determine the best option.

With respect to software boxes, there was no known software package that was

exclusively used to solve expected values and, had there been one, it would probably

have been of a black-box type. Thus, the three software boxes were developed in Excel

spreadsheet using visual basic application (VBA). The black-box application allowed

students to calculate the expected values without showing steps whilst the glass-box

software performed calculations showing the steps for each game. Open-box software

allowed the students to interact with the software at each step for the game particularly

with respect to choosing the arithmetic calculation and in so doing calculating the

answer. All three software boxes were developed in separate Excel sheets (see Figure 10

on p.103 and Figure 11 on p.104). There were three more sheets also developed: 1) the

data entry sheet in which the data could be entered before using the software boxes, 2)

the scrap sheet in which students could do calculations if they wish and 3) the answer

sheet in which students recorded their answers during the post test (see attached CD for

examples of the software boxes).

Black-Box

Figure 10: Screenshot of using the black-box software for solving expected values

As indicated in Section 3.3.2 (p.59) an experimental protocol used by Renkl et

al. (2004) was employed. The background questionnaire was set up as a web-page and

sought to find answers relating to age, level of mathematics acquired, gender,

confidence in mathematics, computers and Excel spreadsheet, and whether they had any

knowledge on expected values. These values were intended to be used as covariates in

the main study analysis. Whilst the recommendation for any covariates, particularly

ones where the participants self-identified their levels should have a series of questions

(e.g. Owen and Froman, 2005), it was felt that subjecting participants to a longer series

of items would require more time from students and may affect the number of students

willing to participate in the study. The instructional materials as mentioned before

provided background information on expected value and were also supplemented with a

Glass-Box

Open-Box

Figure 11: Screenshots of using the glass and open-box software for solving expected values

The pre-test consisted of using simple probability questions which were awarded

one mark each (see Appendix 3, p.294). An example of a pre-test question is as follows:

If a dice is rolled, what is the probability that the dice will have a value of four or more?

The pre-test was based solely on simple probability since Renkl et al. (2004)

suggested using a level of difficulty that was lower than the post-test.

In the post-test, there were 9 tasks (three from each task type): the first 6 tasks

paper or the software box and to provide an explanation for the answer. The answer

sheet in Excel was used for entering the answers for the post-test.

Table 14: Examples of expected value tasks

Mechanical Task: Which of the following games would I get the best expected value for?

Game 1:

1st prize: 47% probability of winning £105 2nd prize: Expected prize of winning £58 Game 2:

1st _{prize: Expected prize £98}

2nd _{prize: 37% probability of winning £129}

Game 3:

1st prize: 78% probability of winning £68 2nd prize: Expected prize of winning £135

Interpretive Task: Which of the following games would I get the best expected value?

r is an arbitrary probability. Give your reasoning.

Game 1:

1st prize: (r-30%) probability of winning £56 2nd prize: Expected prize of £25

Game 2:

1st prize: r probability of winning £55 2nd prize: Expected prize of £25 Game 3:

1st prize: (r + 10%) probability of winning £25 2nd prize: Expected prize £21

Constructive Task: Joan’s assets consist of £10,000 in cash and a £90,000 home. During a given year, there is a 0.001 chance that Joan’s home will be destroyed by fire or other causes. How much would Joan be willing to pay for an insurance policy that would replace her home if it was destroyed?

Following the post-test a short interview was conducted with the students to

elicit their opinions on the three software boxes and on expected values. Each task was

mechanical in nature. All nine tasks were unrelated to each other. Only for the

mechanical tasks, students were expected to solve using the software. For the

interpretive tasks, these were expected to be logically deduced.

This study used 6 students for testing the remote observation process and a

rotational confounded study design (Campbell and Stanley, 1963) was tested, where

each student used the three software boxes in 6 permutations. Students solved all

mechanical tasks first, followed by the interpretive and the constructive tasks. The

students solved one task type each with one software box (see Table 15).

Table 15: The sequence the software boxes were used to solve expected values tasks

Student Tasks 1, 4, 7 Tasks 2, 5, 8 Tasks 3, 6, 9

1 B Open Black Glass

2 J Glass Open Black

3 G Open Glass Black

4 Cl Glass Black Open

5 Ch Black Glass Open

6 R Black Open Glass