Research method and way of work

The research method adopted in this work is action research [90]. In action re-search the construction and evaluation of an approach under development is done in an integrating and re‡ecting context. This means that the constructors (such as developers of software systems or researches of methodology) and end-users work together in a setting that enables the end-users to express their goals and opinions explicitly and implicitly and the constructor can passively observe and actively interact with the tasks performed by the end-users. Here, the goal of the constructor is to understand the why, when and what of the tasks performed by

1.4 Research method and way of work 13 the end-users. The latter is called change and re‡ection and hence the emphasis in action research is more on what practitioners do than on what they say they do.

Action research comes in a number of shapes and are applied in …elds such as organisation development and education and health [90, 99]. Lately, action re-search has also been adapted for and applied in the software engineering domain [100], such as that of the CORAS project [19]. More information and appropri-ate references on action research and theory on community of practice for which action research is based upon are in Munkvoll et al. (2003) [76].

1.4.1 Application of action research for this work

Most action research projects set out to explicitly study something in order to change and improve it. In many cases the need for change or improvement arises from an unsatisfactory situation that those a¤ected want to improve. However, it can also arise from a need to extend or re…ne an already satisfactory situation. As improvement can only be facilitated by a complete understanding of the particular work situation it is important to ensure that the constructors have the ability to study and understand the problems of the end-users. This is best done through practical participation and experimentation. Hence, the process of action research is iterative and usually consists of some variation of the three phases: hypothesis,

…eldwork and analysis and conclusion based on the results from the …eldwork and analysis.

This work has used an example-driven action research approach both for the ini-tial explorative phase and for the development and evaluation phase. The explo-rative phase focused on understanding the needs of the decision maker (end-user) in the context of security solution decisions. This was performed in a number of iterations through execution and observation of security solution decision situa-tions using simulation examples. The development and evaluation phase included the interpretation of the result from the explorative phase and change and

re-‡ection through example-driven testing of alternative approaches and techniques to the problems or challenges discovered. This phase also went through a num-ber of iterations as well as reiterated back to the explorative phase as it became evident that a better understanding of the problems and challenges involved in security solution decisions were needed. These two phases constitute the overall work process of this work. To achieve a structured and repeatable process within each of the two work phases each was called a construction process and structured like a development process as shown in Figure 1.2.

As action research is used in this work the testing phase is extended to include prototyping. Also, as the evaluation is example-driven through testing and

pro-14 1. Research Context

Fig. 1.2.The phases and iterations of the construction (work) process

Develop version n of approach

Test version n of approach in trial n

Evaluation of version n of approach

If approach does not meet requirements, or predefined goals not met: n=n+1

Requirements and predefined goals are met (sufficently met) Develop version n of approach

Test version n of approach in trial n

Evaluation of version n of approach

If approach does not meet requirements, or predefined goals not met: n=n+1

Requirements and predefined goals are met (sufficently met)

Fig. 1.3.Action research and how it in‡uenced the construction process

totyping an additional loop-back phase called evaluate, re…ne and update are performed in all iterations before …nally entering the implementation phase. This ensures that the approach developed is su¢ ciently mature and evaluated before any implementation activity is started. Hence, the work process iterates between all phases, as well as between the testing and prototyping phase and the require-ments phase. This means that several iterations involving the four …rst phases were executed before any full iteration was performed. Figure 1.3 shows how action research in‡uenced the construction process while Figure 1.4 illustrates how the community of practice was built and maintained throughout this work and and gives an overview of the step-wise construction and evaluation strategy adapted in this work.

1.4 Research method and way of work 15

Fig. 1.4.The evaluation and validation process used in this work

As can be seen in Figure 1.4 the communities of practice were built and main-tained using three main constructs: (1) Requirement speci…cation, design, testing and implementation with involvement from both the practitioner and constructor roles where the practitioners execute and the researchers observe. (2) Test and evaluation through practical examples. (3) Evaluation, re…ne and update based on the results from the example runs. An important question in the latter was:

Did the methodology meet its intentions su¢ ciently in terms of feasibility, usabil-ity and applicabilusabil-ity? The results from the evaluation session in (3) are used as input to the next iteration in the construction of the methodology. This process was repeated until the requirements of the end-user were met or the goal of this work was su¢ ciently achieved.

This work re-engineered parts of the ACTIVE e-commerce platform in three full iterations of the action research based work process described above. Each of these iterations are called a study and referred to as study 1, 2 and 3 in Figure 1.1. These studies are simulation examples as they were not carried out in an industrial setting but rather in a research setting. Details are given in Chapter 18.

In addition to the three simulation examples included as part of the action re-search work process three controlled experiments were executed; one using stu-dents at NTNU and two using a mixture of stustu-dents and people with relevant industrial background at Colorado State University, Fort Collins, US. These three

16 1. Research Context

experiments were performed to explore the relation between some of the variables in the trust-based information aggregation schema described in Part 5 of this the-sis. A selection of the results from the …rst experiment is given in Houmb, Johnsen and Stålhane (2004) [49].