The User Perspective

While usually the participants that take part in a WOZ experiment would be regarded as the po- tential users, our goal here was to explore the WOZ method from a researcher’s perspective. In this case, the person running the experiment, i.e. the wizard, constituted our user and therefore had to be analysed in order to understand their distinct requirements. We looked at the user from two different angles. We started with the literature and analysed the types of users that report on WOZ experimentation. We looked at their motivation for using the method and the settings they had applied when doing so. Here, the literature did not only provide enough data to obtain a solid overview on the usage of WOZ prototyping (cf. Chapter 2), but also served as the basis for the second analysis angle, which used an interview study with 20 participating

researchers from academia and industry to uncover some of the aspects of WOZ prototyping that are rarely discussed in publications.

Literature-informed User Perspective

As a first step we re-visited the different studies reported in the literature and analysed what type of researchers might be interested in using the WOZ method. We looked at their motivation for using the method and where they put the emphasis when conducting the experiments. As the related work part of this thesis has already highlighted (cf. Chapter 2), the literature shows a variety of different application areas for WOZ experimentation. While they all share the overall concept of using a human to simulate a system function, the settings between different application areas can vary significantly. Researchers who are interested in exploring emotional feedback, for example, might be less restrictive when it comes to defining the wizard task, than those who want to test certain components of a dialogue system. Our goal was to identify the different possible settings and map them to explicit system functions that had to be supported by the tool.

Elaborating on what we learned from reviewing publications we also thought about poten- tial user groups outside the academic community. While the majority of WOZ experimentation might happen in the course of scientific exploration there is a valid use case for the method in interaction design, where applicants often work in a different contextual setting. Here the presented work conducted by designers in industry offered a first contact point. Additional insight was obtained by the interview study described in the next section. In summary the goal of this literature-based analysis was to obtain a basic understanding of the different user groups and their distinct motivations for employing the method. While some of them had overlapping interests we also found great differences between users. The analysis served as a necessary basis for the following interviews, which aimed at verifying as well as extending the obtained results.

Interview-informed User Perspective

As the literature alone merely served as a starting point for defining user requirements a more elaborate analysis approach was required to obtain more valid results; especially if the gained insight should go beyond simple functionality definitions and instead generate addi- tional knowledge with respect to employing a known prototyping method. In the past re- searchers have applied different methods for building this sort of understanding of an area of interest. Examples include qualitative research approaches such as focus groups (e.g. Morgan [1997]) and field observations (e.g. Barrett et al. [2004]) as well as more quantitative studies such as traditional surveys and experiments. Although through reviewing the literature we had already built up a certain understanding of WOZ prototyping and what it involves, our goal was to expand beyond the obvious aspects. Hence, we used what we had learned from studying and analysing WOZ experiments and then augmented the results by conducting an interview study.

With this study, we aimed to verify the theories we had developed. The study also gave us the opportunity to explore those aspects of WOZ that are usually suspended when it comes to preparing research results for publication. As such, talking to people about their experience with the method served both, as an instrument of confirmation and verification as well as a tool for new discovery. A similar approach was, for example, used by Clemmensen [2004] to study work practices of HCI professionals. Like focus groups, this form of interview-based analysis tries to generate a broader understanding of an area by not only focusing on a dedicated set of questions but rather allowing for the data to evolve from un- or semi-structured discussions. In our case the knowledge generated from studying the literature served as the source for ini- tial questions. The interview progress itself, however, was steered by the interviewee and the direction he/she wanted to go when talking about their experience. Interviews were fully tran- scribed and a formal analysis of the data, aimed at painting a more holistic picture of the WOZ prototyping method and its applications areas, was conducted. In summary, the goal of this interview study was to broaden our initial understanding of the field created by the literature and to fill-in the gaps and open questions that had remained.