Data Analysis Procedure

This section explains the technique and process of data analysis for generating the study results. Both Experiment O ne and Experiment Two produced two types of data: textual and visual. Table 5.2 outlines the descriptions and functions of these two types of data. All verbal data from the co-discovery sessions were transcribed verbatim, while information from the experience diaries was extracted and organised into transcripts. These textual data were used as a principal source of the research findings, whereas visual data (sketches and video recordings) were used to support the analysis and interpretation of the textual data. The analysis process was assisted by ATLAS.ti software (Scientific Software Development GmbH, 2011), which allowed more efficient data organisation and coding.

Table 5.2 Types and Functions of Experiment Data Type of

verbal e xp lanation of their sketch.

In Experiment One, co-discovery transcripts were the primary data for e xp loring how users anticipate their e xperiences with interactive products, and for identifying the characteristics of anticipated user e xperience.

In Experiment Two, co-d iscovery transcripts were used for

investigating the characteristics of their daily e xperiences of using a given digital came ra over a period of three days.

Expe rience diary data we re combined with Expe riment Two’s co-discovery transcripts to explore real user e xpe rience, and to identify its characteristics.

Sketches were used to support textual data analysis.

Video record ings of the co-discovery sessions, capturing participants’

conversations, facia l e xpressions, gestures, and actions.

Observational data were used to support textual data analysis.

The qualitative content analysis technique was employed to analyse the textual data.

Content analysis is a research method for creating valid and replicable inferences from data to the contexts of their use (Krippendorff, 2004). With its ability to identify core consistencies and meanings underlying qualitative data (Patton, 2002), content analysis is aimed at creating new insights, and enhancing understanding and knowledge, of the phenomenon under study (Downe-Wamboldt, 1992; Krippendorff, 2004). The central idea of this analysis technique is to organise large amounts of text into much fewer, manageable content categories (Weber, 1990); this involves a structured classification process of coding and determining patterns or themes (Hsieh and Shannon, 2005).

Thus, the content analysis technique was found suitable for this research, as it was able to manage and categorise a substantial amount of textual data resulting from Experiment One and Experiment Two. More importantly, this technique also supported information extraction and data interpretation that led to new knowledge of anticipated user experience and enhanced understanding of real user experience.

Since the research literature on anticipated user experience is limited, and the study goal was to generate new knowledge, the inductive or conventional content analysis approach was applied to the data from Experiment One. In this approach, the

categories were entirely derived from the data, rather than being built from a preconceived theory (Elo and Kyngäs, 2008; Hsieh and Shannon, 2005).

The data analysis of Experiment Two also used the same approach despite the availability of an extensive number of theories on real user experience. The inductive, rather than the deductive, approach was used because the purpose of Experiment Two was to identify the differences between anticipated and real user experiences, as opposed to testing the existing theories or variables per se.

Furthermore, the researcher intended to remain open to new or unexpected categories. However, the formation of some categories in the data analysis of Experiment Two was based on the results from Experiment One.

The outcome of the content analysis is categories (including relationships among them) that explain the phenomenon of interest (Elo and K yngäs, 2008). Hence, by using this analysis method, the building blocks of anticipated and real user experiences could be discovered through categories emerging from the data. These building blocks, in turn, contributed to the identification of the characteristics of these two types of experiences. Further, by developing relationships among these categories, the researcher could gain an understanding of how users anticipate their future experiences with product use.

Figure 5.3 Data Analysis Process: Experiment One and Experiment Two

The data analysis of Experiment One and Experiment Two consisted of two main phases (Figure 5.3), which were based on the work of Busch et al. (2005), Elo and Kyngäs (2008), and Hsieh and Shannon (2005). The first phase, data coding, began with repetitive readings of each transcript to become immersed in, and to make sense of, the data. The transcripts were then read more thoroughly to begin the open coding process (Strauss and Corbin, 1998). Here, text segments that seemed to hold key

concepts relevant to aspects of user experience were highlighted, and notes or keywords were written. As this activity progressed, headings for codes emerged.

After open coding of three transcripts, the headings were collected and organised into sub-categories and codes. These sub-categories were grouped into broader, higher-order categories according to their similarities or relationships. The categories, sub-categories, and codes were subsequently translated into an initial coding scheme, in which the scope of interpretation for each sub-category was defined.

Using the initial coding scheme, the first three transcripts were re-coded and the remaining ones were coded. The coding scheme was iteratively revised to accommodate new emergent sub-categories or codes, and to make necessary refinements of the existing ones, including their scope of interpretation. In other words, the coding scheme constantly evolved until all data were included and the categories and sub-categories were saturated. The final coding schemes for the data from Experiment One and Experiment Two are presented in Section 6.2.1 and 7.2.1 respectively.

In parallel with the coding process, important information pertaining to the research question and sub-questions was extracted and recorded. Furthermore, during this data coding, information from participants’ sketches (for Experiment One only) and video recordings facilitated the interpretation of the texts, and the selection of their correct codes. This visual information also supported the data transcription and open coding stages. Examples of how visual data supported the textual data analysis can be found in Section 6.2.1.

Once the coding process had been completed, to ensure coding consistency and to enhance the reliability of data analysis, the coding process was repeated several times at intervals of five to eight weeks. The breaks allowed for a fresh perspective, not only to reflect on the analysis process, but also to verify that the coding scheme had been correctly applied and that the coder’s understanding of the categories or sub-categories did not change over the time.

The second phase of analysis was the conduct of relational analysis to explore meaningful relationships among the sub-categories identified (Busch, et al., 2005).

The type of relational analysis used was proximity analysis, which was concerned

with the co-occurrence of the sub-categories or codes in the data (Busch, et al., 2005;

Morse and Field, 1995). Two or more codes that were assigned to the same text segment, or to parts of the text that overlapped, were identified. The co-occurring codes and their associated texts were then explored to develop relationships among the codes. Finally, these codes (sub-categories) and their associations were represented visually via networks or conceptual maps that served to explain the overall meaning of the data (Busch, et al., 2005). In Experiment One particularly, these sub-category networks formed the AUX Framework, which elucidated the way in which users anticipate their experiences with interactive products. In Experiment Two, on the other hand, they represented the user’s process of experiencing an actual product. Chapters 6 and 7 elaborate the data analysis procedure for each experiment.

Following the completion of the data analyses for Experiment O ne and Experiment Two, the results were synthesised and further analysed to differentiate between anticipated and real user experiences. Relevant criteria were determined as a basis for this comparative analysis. An example of these criteria was the importance of pragmatic and hedonic product qualities perceived by users in their anticipated and real experiences. Based on the analysis outcomes, design recommendations were proposed to support early assessment of, and design for, user experience.

5.6 SUMMARY

This chapter highlighted the research methodology and plan that were designed to explore anticipated and real user experiences. The selected research approach and methods have been introduced and justified. These methods included co-discovery, visual representation (sketching), experience diary, and observation. The research plan has also been outlined in its five stages: (1) Experiment O ne, (2) Experiment Two, (3) Comparative analysis, (4) Framework and design recommendations, and (5) Conclusions.

Furthermore, the choice of a specific category of interactive products for this study (i.e. digital cameras) was explained. This chapter then delineated the process of participant recruitment, which covered the sampling techniques employed (i.e.

snowball and volunteer samplings) and the screening questionnaire used. Lastly, the

chapter detailed the data analysis procedure, which consisted of data coding and relational analysis phases.

Chapter 6 now focuses on Experiment O ne, which has been introduced in this chapter. The details of the experiment design and procedure are described, and are followed by an in-depth explanation of the data analysis. The results of Experiment One are then presented.