3 Mixed Methods Implementations
3.1 Triangulation of User Research Methods
Getting users to talk about and explain their experience is the easiest and most widely applied approach to understand ones experience (Albert & Tullis, 2013). However, self-reporting techniques are limited when conducting GUR in comparison to a usability evaluation on productivity applications ((Hazlett, 2008), p. 189). Firstly, self-reporting methods, such as questionnaires and interviews, are sampling methods, meaning that the players will be responding at a specific moment in time. If they fill out questionnaires during the game it interrupts the gameplay/flow and modifies their experience. However, if we wait until the end, then they may have forgotten what the real experience was like, or they may not remember correctly (Ravaja, 2004). Secondly, if we ask players to self-report, although these can
potentially provide a rich source of data, we are relying on their awareness, recall, and cognitive filtering abilities to function before a response emerges, and covertly assess the experience. This section explains three approaches, which are explored as part of this thesis to enhance self- reporting approaches for GUR. These approaches are also used to collect players’ data for BioSt prototypes.
3.1.1 Player’s Self-assessment Diagrams
One motivation for this research is an idea to find a way to capture players overall gameplay experience, without interruption from the user researcher. One approach tried as an exploratory task was to provide players with a blank graph paper and asked them (without interruption or prompting) to ‘draw their experience’ at the end of each level (which provides a natural break in the gameplay experience). The interest was to explore how much detail players could recall
reflect on a player’s overall experience of each level without many details. For example, these diagrams could identify the gameplay issues that the players may tell their friends about. Also these player experience diagrams seem to address the perception issue: what the player thinks happened, and what they can recall.
Figure 3-1 Example of a player’s self-assessment diagram for 30 minutes of gameplay
On the other side, in the vast majority of cases, players could not accurately remember details of their gameplay experience, even after short game sessions. It seems that many players are only able to recall few details from the very beginning and the very end of that gameplay session. In most cases, the players draw a line graph which contains few peaks (or thoughts). In psychology this is known as the serial position effect (Feigenbaum & Simon, 1962). Broadly speaking, people tend to remember events at the start, the end, and perhaps one in the middle. Figure 3-2 shows an example of a player experience diagram that a player has drawn after just a 20-minute gameplay session.
Figure 3-2 Example of a player’s self-assessment diagram showing recalled events at the start and the end These outcomes are all expected due to the limitations of self-reporting approaches as explained earlier. Later this chapter, section 3.3 shows how combining these players’ drawings with
These player’s drawings are useful in a triangulation setting, providing extra evidence to
support (or confirm) findings from other approaches (this is their contribution in creating BioSt).
3.1.2 Player’s Physiological Arousal to Structure Post-session Interview and Coding Gameplay Events
Chapter 2 discussed that the use of physiological measures does not directly identify the feeling that a participant is experiencing. Generally, researchers using physiological approaches may find it difficult to match the obtained quantitative data to the participant’s emotional experience during an experiment (van den Broek, Lis!, & Janssen, 2010). It is also possible to consider that player could be emotionally provoked, not because of specific in-game elements, but as a response to an external activity, anticipation, or something not otherwise observed. The often described ‘many-to-one’ relationship between psychological processing and physiological response allows for physiological measures to be linked to a number of psychological structures (Cacioppo, Tassinary, & Berntson, 2007).
Based on previous research, as discussed in Chapter 2, this thesis assumed a mapping of GSR arousal to player excitement (or frustration). The interest of this thesis is not to explore this mapping (it has been widely explored before). Neither to attempt to map the changes in a player’s physiological measures to a particular emotion, instead using measures of the player’s phasic physiological data purely to log 'micro-events' in the game. For example, a visual change (Figure 3-3) in the player’s arousal level (peak in GSR measurement) are used to bookmark the player’s gameplay video. These timestamps of ‘micro-events’ provide the structure for a post- session interview with the player.
noted during the playtest, constructing a log of times during gameplay in which players experienced a potentially meaningful degree of arousal. Micro-events were not analysed or interpreted at the logging stage, and at no time were individual participant’s GSR measurements compared to other players. Instead, after the gameplay session, the video footage related to every logged micro-event was played back to players, who were asked to recall these specific moments and inform the user researcher of their thoughts. All logged micro-events were
addressed in this manner, with usability and player experience issues determined by the players’ interpretation of their physiological response.
The study S1 (section 3.2) demonstrates the value of this approach in the field of GUR and in generating BioSt. The results from this study show that using physiological arousal to drive post-session interviews would result in more meaningful insights into players’ motivations and expectations.
As discussed in Chapter 2, observing players interacting with the game is one of the common approaches in GUR that provides a rich source of data. Physiological measures can shows the corresponding biological reaction from the player’s body to game events. In addition to structuring post-session interviews based on the player’s physiological arousal measurements, the techniques presented in the previous section form a framework for analysing the coupling between player behaviour (what they did) and feeling (how they felt).
The following sections of this chapter contain a description of two initial studies of this thesis, where I explored the contribution of explained approaches in GUR. The knowledge gained from these studies helped me to create, iterate, develop and investigate the usefulness of underlying methods in order to develop BioSt.