Research Design

4.2.1 RESEARCH QUESTION

Prior to selecting a research strategy, it is necessary to determine the research topic, question, and purpose. The topic of this research is engineering management; more specifically, improving control of design and construction processes on architectural/engineering/construction projects. The questions driving this research are:

1) 1) What can be done by way of tools provided and improved implementation of the Last Planner system of production control to increase plan reliability above the 70%

PPC level? 2) How/Can Last Planner be successfully applied to increase plan

reliability during design processes¹⁸? The purpose of the research is to evaluate and improve the effectiveness of this managerial policy and practice.

Evaluation is a type of applied or action research (McNeill, 1989), concerned with technology in the broad sense; i.e., goal-oriented action. Evaluations typically pursue improvement of the subject policy or practice in addition to rating effectiveness against objectives. Simple rating is often made more difficult because of changes made mid-stream in the policy or practice being evaluated. Opportunity for improvement seldom waits on the desire for an unambiguous definition of what is to be evaluated. Indeed, evaluation and improvement often blur together, especially when the researcher is involved in the creation and implementation of the policies and practices being implemented and evaluated, as is the case with this researcher and research. Some might worry about an involved researcher’s objectivity. On the other hand, it may simply be that technological research demands another concept and procedure than that of traditional, fact finding research.

Evaluation does not fit neatly within the classification of traditional purposes of enquiry; i.e., exploratory, descriptive, explanatory. The conceptual model for technological research appears to have been drawn from the natural sciences, for which the (immediate) goal is rather to understand than to change the world. Policy evaluation involves exploration, description, and explanation, but subordinates those purposes to the overriding purpose of improving practice. Nonetheless, improving practice requires understanding what works and does not work, and to as great an extent as possible, understanding why what works and what does not. Consequently, the purpose of this

18

research includes determining the extent to which the Last Planner system is effective and why it is or is not effective.

4.2.2 RESEARCH STRATEGIES

The three traditional research strategies are experiment, survey, and case study (Robson, 1993, p.40). It has previously been argued in this chapter that a survey strategy is inappropriate for the question posed by this research. The research strategies that could possibly lend themselves to investigation of this research question include true experiments, quasi-experiments, and case studies.

True experiments require establishing a control group that differs in no relevant way from the experimental group. A true experiment was not appropriate because of the difficulty of establishing a control group and lack of control over extraneous variables.

At first glance, it would seem to be possible to use a pre-test, post-test, single group design, measuring flow reliability of the same group before and after implementation of the Last Planner system. This approach has several difficulties: 1) Work flow reliability is not an explicit, measured objective of traditional production control systems, so pre-test quantitative data is not available, and 2) our ability to generalize from the experimental results is limited by the possibility that those who choose to try the Last Planner method are somehow different from those who do not so choose. The second difficulty could be managed by conditioning and qualifying the inferences drawn from the experiment. The first difficulty, the lack of quantitative data on flow reliability for the pre-test, could be handled by substituting subjective data, in the form of interview results. However, this is clearly an inferior solution, and so pushes the researcher to find a more effective research strategy.

Quasi-experiments are “…experiments without random assignment to treatment and comparison groups.” (Campbell and Stanley, 1966, cited in Robson, 1993, p. 98) They admittedly sacrifice some of the rigor of true experiments, but are nonetheless appropriate for a large range of inquiry, where true experiments are impossible or inappropriate. The key issue regarding quasi-experiments is what inferences can be drawn. It is proposed that inferences be justified in terms of study design, the context in which the study occurs, and the pattern of results obtained (Cook and Campbell, 1979).

While this strategy responds to the difficulty of generalizability posed above, it still leaves us without pre-test quantitative data on flow reliability in design, and consequently, is not by itself an adequate strategy for pursuing this research.

Case study is “…a strategy for doing research through empirical investigation of a contemporary phenomenon within its real life context using multiple sources of evidence” (Robson, p. 52). Case studies are an appropriate research strategy when there is little known about the topic of interest, in this case, for example, how production is managed in design; and a change in theory or practice (production control) is proposed (Robson, p.169). Multiple case studies allow the researcher to pursue a progressive strategy, from exploration of a question to more focused examination of trials. Given the policy nature of the research question being posed, a multiple case study strategy seems appropriate.

4.3 Research Methods