Research Approach and Methods

This study employed a qualitative research approach, as it is capable of capturing users’ anticipated and felt experiences and emotions with respect to the use of interactive products. This approach is also useful in answering the ‘how’ and ‘why’

questions pertaining to user-product interactions. According to Strauss and Corbin (1998), qualitative research can refer to studies about people’s lived experiences, feelings, emotions, and behaviours. It enables investigators to explore users’ core experiences, to identify how meanings are created, and to discover (rather than evaluate) variables (Corbin and Strauss, 2008). Marshall and Rossman (1999) concur that qualitative research is interpretative, pragmatic, and grounded in people’s lived experiences. Therefore, qualitative methods can acquire complex information about individuals’ thought processes, feelings, and emotions, which are difficult to explore by using more conventional (e.g. quantitative) methods (Strauss and Corbin, 1998).

In the qualitative paradigm, theories are developed by using inductive reasoning to create a deep understanding of the meaning of data that are gathered from a rich context-bound situation (Creswell, 2003; Silverman, 2005). Thus, qualitative research methods can be used to learn about subjects that have not been well understood, or to gain novel understandings of areas that are already well understood (Stern, as cited in Strauss and Corbin, 1998). In the case of this study, qualitative methods were used to develop new knowledge about anticipated user experience.

In the context of user-product interaction, quantitative methods have been considered less informative for designers than qualitative methods, especially with respect to transforming the findings derived from quantitative data into a product’s features and functions. Kanis, Weegels, and Steenbekkers (1999) refer to previous studies to

demonstrate the limitations of quantitative research in supporting the design of usable consumer products. They conclude that the summative measures resulting from quantitative analysis (e.g. performance time and numbers of errors) provide little information about actual problems in user-product interaction and, consequently, are not particularly useful in generating specific design recommendations.

Kanis et al. (1999) argue that, to inform product design, it is essential to conduct in-depth studies into actual product use in a natural context; this is best achieved through observations enhanced by the user’s clarifications of that use. It is this observational research practice that makes the qualitative approach more useful and enlightening for the design fields. Additionally, Kanis and Green (2000) suggest that there are two criteria for observational research to support usage-oriented design: (1) a focus on user activities when using a product, and (2) a clear relationship between these observed activities and design recommendations for the product’s features and functions. The authors assert that qualitative research is effective in meeting these requirements.

The use of qualitative methods is evident in user experience research. For example, to identify, analyse, and evaluate users’ experiences in using technology or interactive products, researchers have applied the following methods: narrative and storytelling interviews (Geven, et al., 2006; Gruen, Rauch, Redpath, and Ruettinger, 2002; Schrammel, et al., 2008), co-discovery (Jordan, 2000), observation (Väätäjä, 2010), various types of experience diaries (Isomursu, et al., 2004; Karapanos, et al., 2009; Korhonen, et al., 2010b; Swallow, et al., 2005), probe techniques (Jääskö and Mattelmäki, 2003), sentence completion tools (Nurkka, K ujala, and Kemppainen, 2009), and generative tools (including sketching techniques) (Sanders, 2000;

Sleeswijk Visser, et al., 2005; Stappers and Sanders, 2004). Some of these methods have been described in Chapter 2. As well as demonstrating the usefulness of the qualitative approach in user experience studies, this pool of methods provides a source for determining suitable methods for this research.

To address the research question and sub-questions, and to maintain the research rigour, this study utilised a methodological triangulation approach (Denzin, 1989)

that encompassed four qualitative methods: co-discovery, visual representation (sketching), experience diary, and observation. These selected methods were deemed appropriate to achieve the objectives of this research. The justifications for their selection are explained in Sections 5.1.1, 5.1.2, 5.1.3, and 5.1.4.

Each of the study’s experiments comprised a combination of three of the above methods (Section 5.2). According to Denzin (1989), the triangulation of methods is one of the soundest strategies for theory construction. In this approach, several different methods complementarily compensate for the possible limitations and weaknesses of a single method (Flick, 2009; Potter, 1996). For instance, visual data from observation offer participants’ body language information that is not evident from verbal co-discovery data. Furthermore, the meaning of data from one method can be cross-checked with that from the others, thus helping to validate data interpretation. The next four sections discuss the methods used.

5.1.1 Co-discovery

Co-discovery is a user experience exploration method that involves two participants collaboratively discussing a product or concept and its use, while the researcher observes (Jordan, 2000). Communicating with another participant is more comfortable and natural than thinking aloud without a partner (Dumas and Redish, 1999; N ielsen, 1993). The paired participants also have less test anxiety and feel less pressure to express their views, because they do not directly talk to the investigator (Jordan, 2000; Wilson, 2004), as is the case in one-to-one interviews. The co-discovery method, therefore, generates more comments and opinions about users’

thoughts and expectations, and reveals more experiential information related to product use (Dumas and Redish, 1999; Hackman and Biers, 1992; Jordan, 2000).

Hackman and Biers (as cited in Dumas and Redish, 1999) have evaluated the co-discovery technique and verified that it produces valuable data that contribute insight into the design. Zhao, Popovic, Ferreira, and Xiaobo (2007), for instance, have used a co-discovery protocol to explore the future travel needs of elderly Chinese drivers to assist in the design of vehicles for older adults in China.

For the above reasons, the co-discovery method was deemed suitable for exploring anticipated and real user experiences, and was used in Experiment One and Experiment Two of this research. In Experiment O ne, this method was employed to gather rich data about participants’ imagination of a desired product, and about their anticipation of interactions and experiences with the imagined product (Chapter 6).

By analysing the participants’ verbalisations, the researcher gained insight into the user’s process of anticipating experiences with interactive artefacts, and determined the characteristics of this anticipated experience. Meanwhile, in Experiment Two, co-discovery sessions were conducted to obtain retrospective information regarding participants’ experiences, emotions, and perceptions of their actual interactions with a given interactive product (Chapter 7). This information, together with experience diary data, served as a basis for deriving the characteristics of real user experience.

5.1.2 Visual Representation (Sketching)

Sketching or drawing, a type of visual representation technique, was also employed in this study. It has been demonstrated that there is a connection between sketching and experience; thus, the use of visuals can access and portray aspects of user experience (Chamorro-Koc, Popovic, and Emmison, 2008, 2009). This is because people accumulate visual references of their experiences (Chamorro-Koc, 2008). As suggested by Collier (2001), visual information is a source of human experience analysis, through which meanings and patterns can be identified. As research data, every component of a visual record can be a significant source of knowledge when analysed (Collier, 2001).

Previous studies have used visuals not only to explore people’s imagination and perception (Arnheim, 1993; Baskinger and Nam, 2006; Kavakli and Gero, 2001), but also to gain insight into their concepts and perspectives (Chamorro-Koc, 2008;

Sleeswijk Visser, et al., 2005). In addition, visuals have been applied in design research as one of the generative tools to elicit users’ past, current, and future experiences (including memories, feelings, emotions, needs, expectations, and dreams), and to uncover users’ ideas regarding scenarios and contexts of product use (Sanders, 2000; Sleeswijk Visser, et al., 2005; Stappers and Sanders, 2004).

Sketching tasks were included in Experiment One’s co-discovery sessions to better capture participants’ desired product concepts, and to obtain further experiential data, such as pictorial descriptions of perceived experiences, procedures, and situations of product use (Chapter 6). More importantly, sketching also functioned as a means of making an imagined product more concrete, thus assisting participants to anticipate their interactions and experiences with their product concept. Moreover, the use of visuals was helpful in allowing participants to express thoughts that they might otherwise have found difficult to verbalise during the discussion. Information derived from participants’ sketches also supported the analysis of textual data, by helping the researcher in interpreting and coding their verbal responses.

To help clarify meaning, oral explanations (i.e. retrospective verbal report) were a part of participants’ sketching tasks. This step was important in helping the researcher to more effectively interpret the drawings. It also allowed participants to indicate any aspects that could not be conveyed through their sketch (Chamorro-Koc, et al., 2009).

5.1.3 Experience Diary

An experience diary can be defined as questionnaire- like forms that are given to participants for them to record their experiences with a product over a period of time (e.g. days or weeks) (Jordan, 2000). A major advantage of this method is its ability to provide insight into the changes in user experience over time (Jordan, 2000;

Karapanos, et al., 2009). The use of diaries also allows user experience data to be captured from the real contexts of product use (Swallow, et al., 2005).

The experience diary used in this research was based on the Day Reconstruction Method (DRM) (Kahneman, Krueger, Schkade, Schwarz, and Stone, 2004b) that has been adapted and used for user experience research in a longitudinal setting (Karapanos, et al., 2009). The DRM was originally designed for investigating how people experience their various life activities and situations (Kahneman, et al., 2004b; Stone et al., 2006). At the end of a reported day, or at the beginning of the following day, DRM asks participants to recall their experiences by reconstructing activities or events that occurred as a sequence of episodes. Then, for each episode, they are asked to explain the situation and feelings that they experienced (Kahneman,

et al., 2004b). Reconstructing the episodes and contexts of daily activities helps the participants to evoke episodic (specific and recent) memories, thus minimising retrospective bias and errors when reporting on their experiences (Schwarz, Kahneman, and Xu, 2009).

In user experience research, DRM enables users to focus on the perceived product quality within a particular episode of experience, thus reducing the likelihood of reports that are based on their general opinion of the product (Karapanos, et al., 2009). Other advantages offered by this method include: (1) minimum disruptions to users’ normal activities, (2) lower participant burden and a more complete account of activities than is typical for experience sampling methods, and (3) combined assessment of activities and subjective experiences (Kahneman, Krueger, Schkade, Schwarz, and Stone, 2004a; Kahneman, et al., 2004b).

Karapanos et al. (2009) adapted the DRM to explore how user experience develops over time. On a daily basis over a period of several weeks, they asked participants to reconstruct all their activities that related in some way to the product of interest (this procedure is called day reconstruction). The participants were then asked to choose and write stories about the three most impactful experiences of the day (this procedure is called experience narration). These two procedures were adopted in this research.

The experience diary method was employed in Experiment Two to investigate the characteristics of real user experience (Chapter 7). By means of the diaries, participants reported their daily experiences of using a given interactive product for a period of three days. Because they had used it in actual, spontaneous contexts (rather than in a laboratory or controlled situation) over a considerable period of time (rather than a brief interaction), it was expected that participants would report accurate real experiences with the product. Moreover, by adopting the DRM, the participants were likely to avoid describing their general beliefs about the product and their experiences. Reports from the experience diaries were combined and analysed with the co-discovery data to generate the end results.

5.1.4 Observation

Observation involves a systematic and objective process of viewing and recording behavioural patterns and occurrences in a laboratory or naturalistic settings (Tan, 2004). Through observation, people’s complex actions and interactions can be discovered, documented, interpreted, and described (Marshall and Rossman, 1999).

One positive point of this method is that, by evaluating actions in situ, information that reaches beyond individuals’ self- interpretations and personal views about their behaviours and attitudes can be obtained (Gray, 2009). Hence, observational data can be used to supplement and validate participants’ comments.

With respect to participants’ awareness and the researcher’s role, this research used overt, non-participant observation (Gray, 2009). In this approach, the participants were aware of being observed, and the researcher did not work or act alongside them to respond to the experiment tasks. This observation was used as a complementary technique to acquire additional data that might be lacking in participants’ verbal responses, diaries, and sketches. These extra data included, for example, participants’

gestures when expressing their ideas, and their actions when pretending to use an imaginary product. As Flick (2009) asserts, combining or triangulating observation with other sources of data enhances the expressiveness of the collected data.

Actions and behaviours of participants were observed when partaking in the co-discovery sessions (Experiment One and Experiment Two). Video recordings were utilised rather than live observation, so that the experiment sessions could be viewed multiple times in greater detail. This also ensured that nothing was overlooked. The resulting observational information facilitated the interpretation and analysis of verbal or textual data.