Why field studies?

Field trials can offer a high degree of ecological validity. If they are part of the users' real work, the motivation for succeeding will be similar to that in the real work situation. Field trials can also provide, "evaluation conditions [...] similar to those anticipated fo r the actual operation" (Dix et a l, 1993 [p 175]), which was considered likely to increase the reliability of findings.

The main disadvantage is that, in real teaching and learning situations, conditions can not be controlled as closely as in the laboratory. Factors from the real working environment can interfere with the research agenda. In the case of a real teaching and learning context, anything from erratic student attendance to the demands of examinations could cause int^ference.

Experimental studies have the potential for more control. They are attractive where the research questions are very clear cut or where the design has clearly identified targets. The evolution of ClearBoard, as described by Ishii and Kobayashi (1992) is an example. Their design goals are:

"A smooth transition between face-to-face conversations and shared drawing activities [and] Eye contact to support real time and remote collaboration between two users. " [p829]

As a result of comparing systems and observing groups in action, they add requirements for direct drawing on the work surfece and a common orientation for participants, both left-right and up-down, in relation to the work surface. The scope of the study is to find out whether the prototype they create fulfils these requirements. They are thus able to design the expa-iment around a task that enables them to answer the question.

In this case, however, there was no such clear target. It is possible to investigate specific design requirements through laboratory studies but this was also an open enquiry into the role and impact of a shared workspace tool and more suited to investigaticxi in the field.

It can also be difficult to design laboratory investigations of group working. Olson et al. (1992) identify problems in previous studies which might account for contradictory results. Task and group sizes vary and it is difficult to find a large number of "intact'^ groups; the solution is often either to be content with a small number or to create artificial groups. In addition, not enough account is taken of differences in the technology used, for example, the amount of structure it imposes on the activity.

Finding an appropriate task can also present problems in the laboratory, especially if the intention is that findings should transfer to the field. The most convincing experiments choose a task which demands the kind of collaborations they wish to study and are clear about what they need to measure and how to do it. So, for example, Olson et al. (1992) wish to assess both the output of a design meeting and the process. This experiment calls for both computer-supported and unsupported groups, since it is the effect of the support tool on the collaboration which is of interest. They are clear about the scope of the study and take steps to validate the task they choose'*.

Researchers may carry out a series of studies, some in the field and others in the laboratory. The choice usually depends on the research questions and the stage of the investigation. Olson et al., (1992) for instance describes a sequence that moves from achieving a baseline of understanding (field), to study of specific phenomena (laboratory), to assessing technology use in organisational context (field again). In a similar way, Watson (2001) and Watson & Sasse (2000a, b) show how field trials can be combined with laboratory experiments in a research programme to investigate the impact of audio and video quality. Kies et al. (1996) also use a mix of laboratory and field work although their field studies are short in duration.

Another approach is to combine features of field and laboratory work in a single study. Thus, somewhere between field and laboratory studies are those in which conditions in the field are controlled and manipulated over a period of time. Galegher and Kraut (1992) wish to examine a hypothesis derived from contingency theory and related to media choice. Contingency theory suggested that, given a choice of

^ It is assumed that "intact" means "already existing", "having an existence outside the study". * They use McGrath's (1984) Task Taxonomy for task validation.

communication conditions (face-to-face, telephone, document exchange, computer conferencing), fece- to-face will be chosen for the most "communicatively intense" parts of the work. Two tasks, both complex group projects which could be broken into sub-tasks, are given to groups of students who have different sets of media choices. For each project, the sub tasks varied in the intensity of communicaticm they demanded. One project, however, was designed to demand far mwe collaborative activity than the other. Collection of subjective data is integrated into the normal work pattern through a requirement for daily work reports. Since it is integrated into day-to-day study, this has aspects of the field trial, but fer more control has been exercised over conditions than is usually possible in the field. All the same, the writers note the constraints imposed by using students from a real class for an experimental study. The need to be fair, for example, prevents exactly equal numbers of groups fi’om being assigned to each condition. Similarly, Whitelcmk et al. (1995) conduct an empirical study in a schcml environment, exercising considerable control over the conditions. 15-year-old students, assigned to 4 conditions, are given a pretest of conceptual understanding, some exploratory work with a simulator, three problems to solve, and a post-test, a few days afterwards. This is a one-off event but it is embedded in the curriculum and planned in the way that a good lesson would be

In the case of this research, field studies were considered appropriate but the degree of control varied according to the stage of the research. This is explained in the next section.