Research Methods .1 D ATA C OLLECTION

Executing a research strategy requires methods for data collection and analysis. What

pursued in this thesis? Of those available, which fit best with conditions such as accessibility to people and documents, involvement of the researcher in managerial decision making, time available, etc?

Methods for data collection include direct observation, interviews and questionnaires, and documentary analysis. A variant of direct observation is participant observation; i.e., observational reporting by a researcher who is part of the group being observed.

All these methods of data collection are used in this research. In all cases, the researcher served as a consultant to the project team, and consequently was in the role of participant observer rather than a neutral observer. Specific observational data was collected from participation in project coordination meetings and other events devoted to planning and controlling design and construction processes. Interviews or questionnaires were used in all cases to collect team member assessments, both during the course of each project and at the conclusion of each. Interviews were also used to collect other participants’ observations of meetings and events relevant to project control at which the researcher was not present. Records collected included meeting minutes and memos, various forms of schedules, and action item logs. In all cases, measurements were made and recorded of short-term assignments, their due dates, actual completion dates, and reasons for failure to complete assignments on their due dates.

4.3.2 DATA ANALYSIS AND EVALUATION

McNeill (1989) suggests three key concepts: reliability, validity, and representativeness.

Reliability concerns the extent to which research can be repeated by others with the same results. “Validity refers to the problem of whether the data collected is a true picture of what is being studied.” Representativeness concerns whether the objects of study are typical of others, and consequently, the extent to which we can generalize.

Reliability in action research is inevitably questionable because of the active role played by the researcher in generating the phenomena being studied. Validity of findings is especially difficult in survey research because of the potential difference between what people say and what they do. It is less a problem for action research because of its public nature and the availability of measurement data such as PPC (Percent Plan Complete).

Generalizability from the cases is a question that cannot be completely answered, no more than it can for a limited number of laboratory experiments. However, unlike laboratory experiments, policy implementations are made in the messy reality of organizations and social relations. Few if any variables can be completely controlled. In the case of this research, attempts are made to control key variables of implementation and execution of the system. However, it is recognized that control is partial and incomplete. Nonetheless, having demonstrated even on a single project that plan reliability can be improved is sufficient to establish system effectiveness. Future work may be devoted to better understanding the conditions necessary for such success.

Another difficulty is that plan reliability is measured by PPC ('percent plan complete';

i.e., percentage of assignments completed), but PPC does not directly measure plan quality. First of all, success or failure in assignment completion may be a consequence either of the quality of the assignment or of its execution. Since the Last Planner system primarily attempts to improve plan quality, execution failures and therefore PPC may not vary with its effectiveness. In addition, apart from unsound assignments, it is often difficult to differentiate between an execution and a quality failure. Was the assignment poorly defined or was the problem with the lack of effort or skill on the part of the designers or builders?

Yet a further difficulty is the ambiguity of assignment ‘completion’ when assignments have not been well defined. An assignment to “Produce as many piping drawings as you can by the end of the week” might be marked as completed. The researcher can partially guard against this problem by reviewing assignments for adequate definition. However, it is virtually impossible for the researcher to prevent someone marking assignments completed in order to ‘make the worse appear better’. The best defense might be to convince those doing the marking that PPC is not a measure of individual but of system performance. Unfortunately, that is not quite true. Individuals can be better or worse at defining, sizing, sequencing, and assessing the soundness of assignments. PPC records of individual front line supervisors can be revealing of those capabilities.

For these various reasons, evaluating the impact of the Last Planner system on plan reliability is no straightforward matter. Similar difficulties beset improving the system, which occurs through understanding and preventing plan quality failures. It is often difficult to accurately determine reasons for failure. Unsoundness of assignments is the easiest to determine because something is lacking that is needed to do the assignment properly; e.g., a soils report, a stress calculation, a decision between alternative designs, etc. Failures from sizing or sequencing are more difficult to identify. The later case studies incorporate efforts to improve plan failure analysis based on experiences in the previous cases.

4.3.3 CASE STUDIES

The research was done through a series of case studies. The first case, the CCSR project, was an exploratory extension of the Last Planner system to the coordination of multiple trades on a construction project. The primary improvement from that case was the addition of the constraints analysis process. The second case, the Next Stage project, is

an exploratory case study on the extension of the Last Planner system to design production control. Case Three shows the efforts of a speciality contractor, Pacific Contracting, to improve its work flow reliability. It may well reveal the limits on a speciality contractor implementing the Last Planner system unilaterally. Case Four, the Old Chemistry Building Renovation project, shows the potential for improvement in work flow reliability from a more thorough and deliberate education and training of the project team. Case Five is the Zeneca Project, one of several implementations of the Last Planner system undertaken by Barnes Construction with significant education and coaching provided to the participants, and application of the latest thinking and techniques in the Last Planner system.