Method

5.2.1 Participants and Setting

The pilot study took place in one of the labs at the University of Central Lancashire in a MESS day Format where a whole class of 30 children accompanied by their teacher visited the lab and moved between different activities in small groups of five. A typical mess day involves a ‘whole school class’ taking part in several activities designed by several researchers in an ordered manner within the university labs or in their own school (Horton, Read, Mazzone, Sim, & Fitton, 2012). Twelve participants aged between 14 and 16 years old participated in the study. The participants were grouped in pairs by their teacher resulting in two boy-boy groups, two girl-girl groups and two mixed groups. To maintain anonymity of the participants, participant codes (PI-P12) and group codes (A-F) were used.

5.2.2 Research Design

In this study pairs of individuals were observed while they interacted with an ECA enabled game using three game pad, dance mat and tangible controller. A within-subject design approach in which pairs played both the single and collaborative versions of the game using the three controllers (game pad, dance mat and tangible) was used in the study. In order to counterbalance for learning effect, a 3x3 Latin square design approach (Breakwell, Hammond, & Fife-Schaw, 2000) was used to select the order in which every pair played. This resulted in four

79 groups playing the game using game pad first; four groups playing the game using dancemat first and four groups playing the game using tangible controller first.

5.2.3 Evaluation Instruments and Analysis

In this study, a ‘mixed methods’ approach (Tashakkori & Teddlie, 2003) was adopted whereby a range of data collection and analysis methods (both qualitative and quantitative) was used to explore ECA. This approach was beneficial in providing rich findings and answering the research questions of this research and as quantitative or qualitative approaches alone is insufficient. Additionally, the mixed method approach ensured that the limitations of one approach were offset by the strengths of the other. The methods include logging of participants’ interactions with the game, researcher observation of the participants during gameplay and survey methods such as questionnaires and fun toolkit. These methods and how they were analysed are described below.

5.2.3.1 Questionnaire

In this study, both pre-test and post-test questionnaires were used. The pre-test questionnaire was used prior to the start of gameplay to obtain participants’ demographic information such as age, gender and previous experience in technologies and gaming (Appendix3A). It was important to explore the types of games the participants play and the controllers they are familiar with because it was thought that the participants’ prior experience with the technologies could affect the way they collaborate and their gameplay experience. Consequently, the frequency of use (FUS) scale as validated by (Kano, Horton, & Read, 2010) was used to measure how often the participants played computer games on various platforms. The FUS scale was chosen not only because it is used to measure frequency of use but also because it can easily be understood and effectively used by children even those as young as 7 years old. The FUS scale is a 4-point Likert scale and was coded as 4 for ‘everyday’, 3 for ‘a few times a week’, 2 for ‘once a week’, and 1 for ‘never’. The post-test questionnaire was used at the end of each evaluation session to gather the participants’ thoughts and opinions on the technologies. The researcher wanted to find out if familiarity with partner will have an effect on the participants’ collaboration. It was also interesting to know if the participants noticed the map on the screen and whether they knew the purpose of the map. Additionally, the researcher was interested in knowing which version of the game the participants preferred and enjoyed playing. Consequently, the post-test questionnaire contained questions related to familiarity with partner, awareness of the interaction map on the screen and questions on preference for

80 and enjoyment of the game. The participants’ responses to the questions were entered into a table for analysis and simple statistics like counts and percentages were used. Both the pre-test and post-test questionnaires were piloted with two senior staffs in ChiCI group experienced in designing questionnaires for children to ensure they are appropriate for the age group.

5.2.3.2 Fun toolkit

The Smileyometer has been successfully used in studies that investigated children’s enjoyment of interactive products (Read, 2007; Sim, Macfarlane, & Horton, 2004; Sim, MacFarlane, & Read, 2006; van Dijk et al., 2012). The Smileyometer was used in this research to obtain participants’ subjective fun experience after playing the game. It was coded in an ordinal way on a 5-point scale, where 5 = ‘brilliant’, 4 = ‘really good’, 3 = ‘good’, 2 = ‘not very good and ’1 = ‘awful’. Again- Again table was used to measure the participants’ willingness to play the game again. Again- Again table has been shown to be a valuable tool to use alongside the Smileyometer especially in evaluating with children aged 10 years old and over (van Dijk et al., 2012). The Again-Again results were coded as 3 for ‘yes’, 2 for ‘maybe’ and 1 for ‘no’. The Funsorter is a tool devised to encourage children to rank items against one or more constructs in order to record children’s opinions of a technology or activity (Read & MacFarlane, 2006b). The constructs used in this study are ‘most fun’, ‘easiest to play’ and ‘liked the most’. Read highlighted the importance of using picture cards which represents what the participants can understand (Read, 2007). Consequently, in this study the participants completed the Funsorter using picture cards with icons of controllers they had played the game with during the study sessions (Figure4-1). The Funsorters completed by the participants were coded as 3 for the highest ranked, 2 for the next and 1 for the lowest for each of the construct.

Figure 5-1: Picture cards

5.2.3.3 Observation

Observational method was adopted in this study to explore the participants’ collaborative behaviours exhibited during interaction with the ECA enabled game (OBJ1). The method

81 involved the researcher’s observation of the participants whilst playing the game with the three different controllers and taking notes of the strategies adopted during interaction as they unfold. This approach was useful as it provided a quick way to gain insights into the strategies adopted by the participants during game play. The data collected during the study were analysed with another researcher experienced in coding qualitative data to reduce bias. The researchers adopted an inductive thematic analysis method(Braun & Clarke, 2006) where the observational data (notes) were unanimously coded and categorized into themes (i.e. the researchers coded the data separately and the codes obtained were compared and agreed upon). The themes obtained were taken to a third party (senior researcher) for validation. 5.2.3.4 Logging of participants interactions

In this study, every participant’s key presses (left, right and fire), the time in real-time each key was pressed and the date were logged (Table4-1). It was observed that the log files were not designed in a way to provide useful data from the key presses. Hence, some vital parameters such as times when pairs were in agreement or disagreement etc. which are necessary for analysis required in this study were missed out. This problem was corrected in the second pilot study as shown in Chapter 5 section5.2.3.3.

Table 5-1: Extract of participant log file

Date Player Action

26/3/2013-12:43:18:Player 1 moves right 26/3/2013-12:43:19:Player 2 shoots a bullet 26/3/2013-12:43:19:Player 1 moves right 26/3/2013-12:43:19:Player 2 shoots a bullet 26/3/2013-12:43:20:Player 1 moves right 26/3/2013-12:43:20:Player 2 shoots a bullet 26/3/2013-12:43:20:Player 2 shoots a bullet 26/3/2013-12:43:20:Player 1 moves left

5.2.4 Procedure

The study began by each pair completing a pre-test questionnaire on their experience in technologies and gaming. The participants were then asked to stand in a marked area two meters away from a screen onto which the game was projected. This ensured that the participants’ positions from the screen remained the same across groups. The game was projected on a large screen to afford easy interaction with game units and provide large

82 interactional space to promote collaboration among the participants. The researcher then explained the rules of the game and that the game would be played using three different controllers. For training purposes, each participant in a pair played the single player version individually for 30seconds before playing the collaborative version once for two minutes. Each pair completed an evaluation form consisting of the post-test questionnaire, Smileyometer and Again-Again Table (Appendix3B), one per participant at the end of each session to capture their opinions on the technology The evaluation form also contained questions related to collaboration, familiarity with partner, awareness of the interaction map that was included on the screen, preference for and enjoyment of the game. Afterwards, each pair completed a Funsorter (Read et al., 1999) based on which controller they ‘liked the most’, ‘was most fun’ and ‘was easiest to play with’ on the scale best to worst using the picture cards as shown Figure4-1 in section 4.2.3.2. The researcher observed the participants all through the entire sessions and attempted to engage them in informal discussion about how they reached agreement at the end of each session.