Introduction
The goal of this study is two-fold:
a. To determine whether instructional games augment learning.
b. To find out the impact of the challenge and fantasy features on learning in instructional games.
As mentioned earlier, the theoretical framework for educational games is based on
constructivism. One of the main tenets of constructivism is that students construct their own knowledge. As Bruner (1966) mentions:
The will to learn is an intrinsic motive, one that finds both its source and its reward in its own exercise. The will to learn becomes a 'problem' only under specialized circumstances like those of a school, where a curriculum is set, students confined and a path fixed. The problem exists not so much in learning itself, but in the fact that what the school imposes often fails to enlist the natural energies that sustain spontaneous learning. (p. 127)
It is evident from Bruner that instructional games should motivate students to construct their own reality. In the previous chapter, it was established from instructional game related literature that the most important features or design elements were identified as challenge, fantasy and
feedback. Among these three features, feedback was identified as an important factor that contributes towards effective learning and retention. While feedback is considered an important feature, in this research, feedback is held constant for all versions of the video game created for this study. Since the positive impact of feedback on learning has already been well established in existing literature, feedback was not considered as an independent variable in the current study.
More specifically, following were some of the reasons for keeping feedback as a constant feature in the game that was created for this study:
1. There is sufficient literature to support the role of feedback in an instructional environment. Scholars who have done extensive research on feedback mention that any feedback (Wager & Wager, 1985; Kulhavy & Stock, 1989; Johnson & Johnson, 1993; Sales, 1993; Rieber, 1996a & 1996b; Dickey, 2005; Kulik & Kulik, 1988; Dirksen, 2010) positively augments learning and retention. For that matter, even negative feedback enhances learning retention (Carroll & Swain, 1993; Ellis, 2005). 2. By completely eliminating feedback from the game, the game would become artificial
and that would be considered as a threat to the external validity of the study. 3. Another important question to ask is whether we can eliminate feedback in a
computer based video game. Dirksen (2010) says that feedback comes in the form of sounds, visuals, story narration, and interactions with characters, points and rewards. So we can argue that there cannot be a video game without the feedback feature. It is important to note that we can reduce the two independent variables challenge and fantasy to a bare-minimum level and still consider the resulting version as a
videogame. If however we try to reduce feedback to a bare-minimum level, we would no longer have a video game as any form of interaction with the video game system is considered a feedback (Dirksen, 2010).
Therefore the study examines the impact of fantasy and challenge in instructional games on learning while holding the feedback feature as a constant.
1. Research questions and hypothesis
2. Theoretical model and research variables defined 3. Study design explained
4. Context of the study
5. Intervention, i.e. Humatan game explained 6. Pilot study
7. Data collection
8. Data preparation and logging 9. Data analysis
Research questions
The broad question that this study aims to answer is: what are the important features of an instructional game that make it entertaining as well as educational? Since fantasy and challenge are known to be effective features in gaming from previous literature, it is important to see their effect on learning in educational games.
More specifically the questions are:
1. Is there a correlation between the pretest score and the posttest score?
2. What is the impact of including challenge as a design feature of an educational game? • In a statistical framework, this question can be stated as: Are there significant mean
differences for achievement scores between students who play games with the challenge feature turned on and those with the challenge feature turned off? This is one of the main effects that the researcher would like to find from the study.
• In a statistical framework, this question can be stated as: Are there significant mean differences for achievement scores between students who play games with the fantasy feature turned on vs. those with the fantasy feature turned off?
The above two questions denote the main effects that the researcher would like to find from the study. The above questions look at these main effect individual. To test each of these, only the given feature is turned on.
In addition to testing for the effects of each factor individually, a two way ANOVA will evaluate mean differences that are the result of unique combinations of levels of the two factors, fantasy and challenge. The study also tries to answer the following additional questions:
4. Statistically can we state whether or not there is a significant interaction on achievement scores between students who play games with both the fantasy feature turned on and the challenge feature turned on?
The gender, GPA, game skillfulness, and pretest scores are also variables that might influence or be related in some way to the gain score, which is the dependent variable of the study. Hence, these are considered as covariates in the present study. It follows that we must account for the effects of these covariates before we try to find the main and interaction effect of challenge and fantasy. Therefore, the following additional questions will be answered from this study:
5. Are there significant mean differences for achievement scores between Male and Female students?
6. Are there significant mean differences for achievement scores between students with high GPA and low GPA?
7. Are there significant mean differences for achievement scores between students with high SES and low SES?
8. Are there significant mean differences for achievement scores between students with different levels of students’ perceptions about their game skillfulness?
9. Are there significant mean differences for achievement scores between students of various ethnic backgrounds?
To answer the main research question of the study regarding game features, there were three hypotheses stated in the study and it is important to note that these hypotheses are stated as alternative hypotheses. But it was recognized that the statistical testing was conducted against the null hypotheses.
H1: A game that has challenge will positively impact learning.
H2: A game that has fantasy will positively impact learning.
H3: A game that has both challenge and fantasy features will have a higher positive impact on learning than a game with one of these features as well as a game with none of these features.
Theoretical model and research variables defined
In this section a theoretical model for various game features and other variables and their relationship to learning in an instructional game is shown in Figure 3. In this section, the model is presented followed by an explanation of the model. Later, each variable will be discussed in detail.
The theoretical model of game features and other covariates that influence learning is shown in Figure 3. In this model the dependent variable is the gain score (posttest score minus
pretest score). Improvement in learning was measured in this study using the gain score. The alternate hypotheses suggest that video game features such as challenge, fantasy, and feedback should positively impact learning. Along with these features, there are learner characteristics that could be confounding variables that may also impact learning: gender, grade point average (GPA), socio-economic status (SES), game skillfulness, and prior knowledge. A pre-test was administered to assess the prior knowledge, before the intervention. These variables are explained in more detail below:
A. Dependent Variable
The dependent variable is the measure of the learning outcome. In this study, the measure used is the gain score. Gain score is the difference between the posttest and the pretest scores. The posttest score and pretest scores are obtained by summing up the identification test score, terminology test score, and the comprehension test scores. All the six tests (three pretests and three posttests) are attached in the Appendix A - F. These tests are designed to
Fantasy Elements Levels of Challenges Gain Score indicating learning outcome Previous knowledge inferred from the
Pretest score Student’s GPA Gender Game Skillfulness Elaborative Feedback (Kept as a constant in the present study)
Figure 3 – Game features and other covariates that influence learning
measure the overall achievement of the learning objectives presented in the curriculum followed by the Baltimore County Public Schools (BCPS).
B. Independent Variables: Video Game Features
The features of video games and their importance were examined in the previous chapter. Based on the previous literature and scholarly work, and the theoretical framework established by the constructivist’s school of thought, this study examines the following as independent variables: challenge, fantasy and feedback. The importance of each of these is defined below. The operationalization of these variables will be discussed later in this chapter.
Feature: Challenge
Whether in a game or in an educational setting, challenge motivates people. As discussed in the literature review chapter, games that include any form of challenge including conflict, competition, and/or opposition give gamers an adrenaline rush (Prensky, 2001). Gamers love such challenges. Challenge makes instructional games interesting as well as
pedagogically effective. Feature: Fantasy
Scholars say that fantasy plays an important role in a child’s development. It positively supports language and social skills through role-playing (Cassel & Ryokai, 2001). In an edutainment setting, meaningful and appealing fantasy contexts enhance users’ interest in playing the game while also promoting learning (Parker & Lepper, 1992). The literature review chapter of this dissertation discusses several scholarly works that support the role of fantasy in an educational game.
Feedback
From existing literature, it is apparent that feedback is a very important component for better learning (Wager & Wager, 1985; Kulhavy & Stock, 1989; Johnson & Johnson, 1993; Sales, 1993; Rieber, 1996; Dickey, 2005; Kulik & Kulik, 1988; Dirksen, 2010), and that even verificative feedback messages (simple yes/no) are a subset of elaborative feedback. Because any feedback is found to be effective it will be kept as a constant in all the variations of the Humatan game. Capra’s 1996 causality of a feedback loop for
Humatan level 2 is represented here for the present context in Figure 4:
C. Confounding Independent Variables
The following variables might influence or be related in some way to the gain score, which is the dependent variable of the study:
I. Gender: As discussed in chapter 2, there is a discrepancy between male and female students in their attitude towards video games in general. In a study conducted by Hartmann and Klimmt (2006), it was found that girls and women are less involved with video games than are boys and men, and when they do play, they often prefer different games. However recent indications from the industry give a slightly different picture. According to the entertainment Software Association (ESA), the number of girls vs. boys as video game players is changing in the recent times. According to ESA, women constitute 42% of all game players. Women over the age of 18 represent 37% of the game playing population, whereas boys of age 17 or younger represent only 13% of the game playing population (The Entertainment Software Association, 2011). This study will test whether the Humatan game has the same effect on learning for both male and female students. Hence gender is
considered as a confounding independent variable and is coded 0 = Female; 1 = Male. II. GPA: GPA is commonly used as a measure of academic performance (Spitzer, 2000).
GPA is also linked to academic self-efficacy, intrinsic motivation and self-regulation. Therefore, GPA is considered as one of the covariates, since literature suggests that students who have a good GPA may tend to perform better overall.
III. Game skillfulness: Game skillfulness refers to the participants’ perceptions of their game skills and familiarity with different video game consoles. The survey (Appendix G) asked students how skillful they were at playing video games. Since research
interest in the field of instructional gaming is rather recent, there were no surveys available in the literature which could be adapted for the present study. Hence the researcher prepared a survey instrument, and this survey instrument was sent to the Survey Research Lab at University of Illinois at Chicago for review. The final version of the survey instrument was based on the feedback received from the Survey
Research Lab of University of Illinois at Chicago. Game skillfulness is an ordinal measure with three categories of low skilled gamer, moderately skilled gamer and highly skilled gamer. The calculation of this measure will be described in more detail later in this chapter.
IV. SES: The Socioeconomic status (SES) information refers to the percentage of students enrolled in the free and reduced meal scheme offered by BCPS. This data was provided by BCPS.
V. Ethnicity: Students were asked to provide their ethnic background in the survey. This study considers the effect of these confounding independent variables on gain scores in finding out the main effects and interaction effects of the primary independent variables. Any variable that theoretically correlates with the dependent variable in a research study should be considered as a covariate (Stevens, 2001; Mertler & Vannatta, 2010). A covariate is an
independent variable that is not controlled by the researcher but affects the dependent variable. The model described here contains a dependent variable (gain score), two independent variables (challenge and fantasy), and five covariates (gender, GPA, SES, game skillfulness and ethnicity). In this study, data was collected to determine how the independent variables and covariates impact the gain score. In the next section of this chapter the following aspects are explained:
1. Study design
2. Context of the study
3. Intervention: Design of the game “Humatan”
4. Operationalizing the independent variables: Game features 5. Operationalizing the confounding independent variables 6. Data collection
Study design
This study is a univariate. A univariate study has only a single dependent variable. The univariate case of ANOVA is a hypothesis testing procedure that simultaneously evaluates the significance of mean differences on a dependent variable between two or more treatment conditions or groups (Agresti & Finlay, 2009). In this study the single dependent variable is the gain score, the indicator of the learning outcome. The treatment conditions or groups are defined by the various levels of the independent variable in ANOVA terminology. Studies that include more than one factor or variable are known to follow a factorial design. In this study, the researcher has included the following independent variables:
1. Game features (has four levels: challenge on, fantasy on, both on and none on), 2. Gender (has two levels: male and female)
3. GPA (has four levels based on BCPS GPA classification: low, medium-low, medium- high, high )
4. SES (has two levels based on BCPS SES data: middle and high) 5. Game skillfulness (has three levels: low, moderate and high)
6. Ethnic origin (has six levels: African, Asian, European, Hispanic, Middle-Eastern, Native American)
To find the main effects of the game features, a univariate analysis was used. To find out the individual differences the study used the X x Y factorial design model in this study. A typical factorial design is represented in Table 1 shown below (based on Trochim, 2000):
Table 1 – Factorial design representation
R
X11 O
X12 O
X21 O
X22 O
R= Random Assignment; X=Intervention; O=Observations or measures
Table 2 illustrates the levels for the independent variable feature and the covariate gender. Table 3 shows how these levels and factors combine to form a factorial design based on Trochim’s idea.
Table 2 – Levels of Independent variable feature and confounding variable gender IV: Factor A: Feature
Level 1 Challenge feature turned on (Fantasy off) Level 2 Fantasy feature turned on (Challenge off) Level 3 Both features turned on
Level 4 None of the features turned on
IV: Factor B: Gender Level 1 Female Level 2 Male
These two factors form a 4X2 factorial design, as shown below in Table 3:
Table 3– Factors and their levels
Gender (Factor B)
Features (Factor A)
(Level 1) Female (Level 2) Male (Level 1) Challenge on Gain Scores Gain Scores (Level 2) Fantasy on Gain Scores Gain Scores (Level 3) Both the features
turned on
Gain Scores Gain Scores
(Level 4) None of the features turned on
Gain Scores Gain Scores
Factorial design not only tests the significance of group differences (due to the levels of the independent variables), but also tests for any interaction effects between levels of independent variables (Mertler & Vannatta, 2010). In the current context, factorial design not only tests how challenge and fantasy features affect the learning outcome, but also tests the combined effect of gender, GPA, SES, game skillfulness and ethnic origin on the learning outcome measured by the gain score.
Context of the Study
The research was conducted at the Baltimore County Public Schools (BCPS). The BCPS Director of Science and Technology Program agreed to carry out the study in the high schools within BCPS. There are twenty five high schools within BCPS. Not all the schools follow the same state curriculum. The Humatan game was designed to incorporate the following standards:
1. Para-medical Biology: The Human Body (standard 20.4)
2. Human Anatomy: Support, Protection and Movement (standard 12.3)
Those schools which followed either or both the above standards were identified for the current research by the school district’s Office of Science PreK-12. A total of ten schools were selected out of the twenty-five high schools by the BCPS. A detailed description of the sample will be provided later in this chapter.
Intervention: Design of the game “Humatan”
Humatan is an instructional game where high school students will learn to identify and assemble human skeletal structures. This game is specifically developed to address the following standards of Baltimore County Public Schools (high school) curricula:
1. Para-medical Biology: The Human Body (20.4)
2. Human Anatomy: Support, Protection and Movement (12.3) of the BCPS Curriculum
Storyline of the game
It is 1369 BC. King Akhenaton is Egypt’s ruler, with Queen Nefertiti by his side. They don’t have any children yet. A learned scribe has told the King that the ancients had been aware of this and had, with their immense skills, created two little human bodies, that of a male child and a female child, with all the body parts separated and hidden away in the royal palace rooms long ago. This scribe knew how to pray to Lord Anubis to bring the children back to life
provided they could find all the body parts hidden in the palace and assemble them together.
It is now up to the player to find all the parts, get them to the court room, and assemble them correctly to make the new heir to the throne come alive. The player is made aware that during the quest there could be all sorts of danger lurking everywhere.
On the opening screen of the game (Figure 5), the player is given the storyline (please see more on turning the fantasy “On/Off” for the variations of the opening screen) and asked to board a ship to travel in time to reach Egypt in 1369 BC.
Figure 5 –Humatan Opening Screen
Game levels
There are three levels in the Humatan game. These levels are described in detail below. Level 1
When a player arrives at the palace of King Akhenaton and Queen Nefertiti, the player is