Project Performance Management System: A Conceptual Framework for early warnings
in projects
Ingrid Spjelkavik 1
(Senior scientist, SINTEF Technology and Society, Trondheim, Norway) Bjørn Andersen 2
(Professor, NTNU, Trondheim, Norway) Lars E. Onsøyen 3
(Research scientist, SINTEF Technology and Society, Trondheim, Norway) Tom Fagerhaug 4
(Assistant professor, NTNU, Trondheim, Norway) Helge Marheim 5
(Senior Project Manager, Telenor, Oslo, Norway)
Abstract
Projects seem to face problems resulting in cost overruns, behind schedule or poor quality. In this article we introduce a conceptual framework for early warnings in projects as a tool for contribution to project success. The research and development behind this article is based on an iterative process with elements of literature studies, case studies, and on reflection in groups consisting of academics and experienced project managers. The essential elements in the conceptual framework for early warnings are mindset, data-collection, analytics and reflection. Experiences from two companies testing parts of this conceptual framework in some of their projects indicate that the concept of early warning, even when operationalized through relatively simple tools, contribute to project success. We can already conclude that early warnings in projects seem to give attention to key areas in an early stage, which is considered as useful in projects.
Keywords
Early warning, projects, project success, conceptual framework, pilot studies
1 Introduction
Projects, in terms of large scale investment projects, often seem to face problems according to newspaper headlines. Cost overruns, behind schedule and poor quality are three major problems in projects. Merrow (1988), Miller and Lessard (2000) and Flyvbjerg, Bruzelius and Rothengatter (2003) have stated this fact empirically. From our experience we believe projects, no matter size and industry, face challenges of different kinds resulting in cost overruns, behind schedule and poor quality, although mostly the large ones are of interest for the newspaper headlines. To ensure project success it seems like traditional project management is not sufficient and there should be additional tools available in project management.
In this article we introduce a conceptual framework for early warnings in projects as a tool for contribution to project success. By establishing a system for early warnings the project team is able to get signals on emerging problems – including opportunities – and by using these signals is able to act upon them to reduce risk or to improve output.
The development of a conceptual framework for early warnings in projects is based on performance measurement principles and on previous work on early warnings and early signals. This article presents the elements in the conceptual framework and results from two companies testing parts of this conceptual framework in projects. The
1
Corresponding author: Ingrid Spjelkavik, SINTEF Technology and Society, S.P. Andersens vei 5, NO-7465 Trondheim, Norway. E-mail: [email protected]
2
NO-7491 Trondheim, Norway. E-mail: [email protected]
3
NO-7465 Trondheim, Norway. E-mail: [email protected]
4
NO-7491 Trondheim, Norway. E-mail: [email protected]
5
development of a conceptual framework for early warnings in project is a work in progress, and this article presents the results at this stage in the research process.
By the term early warning we use the definition of Nikander (2002):
An early warning is an observation, a signal, a message or some other item that is or can be seen as an expression, an indication, a proof, or a sign of the existence of some future or incipient positive or negative issue. It is a signal, omen, or indication of future developments. (Nikander, 2002:49)
The conceptual framework of an early warning system is supposed to supplement the ongoing data collection work by emphasizing certain areas for monitoring. This indicates use of already existing data as well as collecting new kinds of data.
2 Methods
The development of a conceptual framework for early warnings in projects has evolved over some time. The work is based on needs from practitioners combined with adjusting knowledge in performance measurement and
performance management into a project context. The research and development process is based on an iterative process with elements of literature studies, case studies from interactions with two companies exploring the thinking in their projects and on reflection in groups with both scientists and experienced project managers.
3 Conceptual framework
A Project Performance Management System (PPMS) is not a computer based program to be used in projects, but is a mindset or a way of thinking, a structured way of collecting key data (indicators) presenting the project and to be used for reflection in a structured way in the project by the project team. Mindset, data-collection, analytics and reflection are essential elements in PPMS. These elements in PPMS combine technology with human skills and experience which are organized in a structured manner to provide early warning in projects.
When looking for potential indicators serving as early warning signals in projects, we suggest two principal rules: 1. Measure early in a chain of cause-and-effect – focus on leading indicators.
2. Measure project factors other than the traditional time, cost and quality.
Cause-and-effect (cause-and-problem) chains in projects have been discussed in the literature (e.g. Kerzner (1995), Lientz, Bennet and Rea (1995), Nikander and Eloranta (1997; 2001) Nikander (2002)).
Traditionally, performance measurement in projects has been preoccupied with simple measures of time, cost and quality. However, these factors are consequences of activities, incidents and other conscious or non-conscious actions, or even lack of actions. For the purpose of early warning in projects there must be additional measures that can provide more relevant information. In the search for indicators that can serve as early warning signs in projects, we have turned to sources describing factors of project success and failure. Project management literature contains descriptions of so-called project success factors, or sometimes their inverse, project pitfalls. This topic has been also extensively researched by for instance Fortune and White (2006) and Nikander (2002) who listed a number of key publications on the topic. Important work includes Pinto and Prescott (1988); Kerzner (1987); Baker, Murphy, and Fisher (1983); Pinto and Slevin (1987; 1988); Pinto and Kharbanda (1995); Lewis (1993); Cleland (1994); Harrison (1992); Lim (1987); and Miller and Lessard (2001). A similar synthesis of the literature concerning success factors was made by Torp, Austeng and Mengesha (Torp, Austeng, Mengesha, 2004, in Torp, Magnussen, Olsson and Klakegg, 2006).Similarly, on project problems or pitfalls, some relevant material can be found in Nikander and Eloranta (2001) and Nikander (2002), where compilations of typical project problems were presented.
The conceptual framework for the PPMS is based on adjusting knowledge and principles from performance management into the context of projects. In the following section we briefly describe some important elements in this concept. PPMS is supposed to be a tool for the project manager or the project management. PPMS will support them with leading indicators pointing towards factors more relevant for the purpose of early warning than time, cost and quality. The areas to measure can for instance be communication, interaction, cooperation, troubleshooting, project team and organisational climate. Other areas to measure can be handling of uncertainty and risk
support as some examples on key areas for early warnings. The project has to find areas relevant for the specific project.
The data measured can in some cases be found in existing data material or there must be collected additional data for early warnings. One way of handling the lack of relevant data in the organisation is to rely on surveys made for the project’s early warning purposes. This can give relevant data for early warning purposes, and is a fully acceptable way of collecting data. One important element in PPMS is to focus on key information and keep the number of indicators low. The data must be presented in a pedagogical way and made easily available for those involved in using them. The data is not the results themselves, but in PPMS they work as input for dialog, reflection and analysis of the data. The data must be used on a regular basis as the normal routine in the project management. Otherwise the handling of the data undermines the quality and willingness to give reliable data in the surveys. The main principle in PPMS is that it should not be a resource-demanding activity for giving early warnings. The collecting part should be less resource-demanding than the handling of the information. And the PPMS should give more gains then costs of operationalizing the PPMS. The PPMS should be as a smoke detector in the project, a cheep and safe way of giving crucial information.
4 Description of cases
The first case is the use of an early warning system called „Health Check“ and has been tried out in Telenor. Telenor is a major telecom company in Norway. In the Norwegian operation Telenor is running about 100 projects a year. The „Health Check“-tool is a service offered to project managers in Telenor Norway, in order to assess the health status of the project in specific areas as a tool for focusing on general risk areas. The method is based on a questionnaire consisting of 10 sections (see Table 1) with a total of 20 questions and the questions are based on Pinto and Slevin’s success factors (1988). The questionnaire is completed anonymously by the members of a project’s core team every four weeks throughout the project. A graphic report displays the result – both as a
snapshot for the last survey and as a histogram for the development over time. The report is for the project manager, and it is up to him/her to decide how to use the results. The tool was tried out in two pilots over a period of 18 weeks as a proof of concept. At present four Telenor projects are utilising the „Health Check“-tool and has used the “Health Check”-tool for another 12 weeks.
The eight critical areas for the Tyrihans project
The ten main areas for the Telenor “Health Check”-tool (based on Pinto and Slevin, 1988)
Competence and staffing, i.e., both competence levels and “fill rate” of positions in the project
Project mission, i.e. is the project mission defined or should it be changed
Project participant contentment Top management support Interface actions, i.e., the development of
open and closed issues identified at the interfaces between technical disciplines
Project schedule and plans, i.e. regarding realistic schedule and plans
Inspection actions, a similar monitoring as interface actions, here of issues identified through various types of inspections
Client consultation, i.e. communication with the client and his needs to be solved by the project
Uncertainties, i.e., the status of existing and new uncertainty (risk) elements
Personnel, i.e. project participant contentment
Contract growth Technical expertise; i.e. regarding competence and staffing
Changes with cost consequences Client acceptance
Monitoring and feedback; i.e. handling of project progress
Communication; i.e. flow of information internally and externally
Flow available along the critical path of the project
Troubleshooting, i.e. handling trouble Table 1: The major areas in the Project Performance Measurement Systems in the pilot studies
The second case project is an oil field development project called Tyrihans, performed by Statoil (later StatoilHydro after a merger with Hydro). It is a technically complex project with a budget of about 1.8 billion €. The project relies on extensive use of external contractors with as much as 80% of the budget being consumed by external
participants. The project manager of Tyrihans early identified a need for an approach that would allow him to “be on top” of the project. Traditional progress measurement was assumed to both hide potential problems and warn of trouble too late, so he was looking for an early warning “smoke detector” system for the project. The authors of this paper were actively engaged in developing such a system, both in terms of indicators to monitor and how to use them during the execution of the project.
The required data for the Tyrihans indicators were partly collected from existing data in various management systems and partly through dedicated data collection based on surveys. The information was presented using different formats of graphs and diagrams, also employing colour codes (traffic light methodology) to indicate status. At regular intervals, the measurements were discussed in the various teams inside the project, to detect any negative developments, pre-emptively address issues under development, and generally create a common understanding of the state of specific areas in the project.
5 Results and experiences
The testing of the “Health Check”-tool in projects in Telenor is still in an early phase. As the preliminary results from the testing are positive Telenor has implemented the „Health Check“-tool as a compulsory tool for all
big/complex/strategic projects in the division Telenor Norway and as a optional service for other projects. After the pilot phase the “Health Check”-tool was evaluated, showing that 80 % of the respondents spent less than 5 minutes answering the survey, and 90 % experienced increased attention on risk areas in the project as a result of the “Health Check“-tool. Discussions with project managers experiencing the “Health Check”-tool indicate that the most interesting aspect of the „Health Check“-tool is that it expresses intuitive uneasiness earlier than the project management resources are capable of identifying the definite risk of the kind the tool focuses on. In this way the “Health Check”-tool acts successfully as a tool for early warning in projects. In the evaluation of the “Health Check”-tool in Telenor the focus has specially been put on the effect/cost-ratio – the method gives significant input (early warnings) to the risk identification and has a very low cost for using it. Experiences from the pilot testing indicates that the use of the tool and the attention the tool attracts to specific areas seems to generate a focus and a consciousness of risk management in general and has contributed to improving the risk culture in Telenor projects. After having implemented the early warning measurements in the Tyrihans project, their use was observed through a period of three months. The main findings from the observation were:
The approach clearly seemed to serve the intentions of providing a regular “temperature reading” of critical factors in the project enabling a more pre-emptive project management approach by giving early
indications of impending or emerging problems.
Furthermore, the use of such measurements and the communication of the results to a wide group of project participants has ensured a more open style of communication and created a better sense of shared
objectives and team spirit.
The evaluation revealed that seven of the eight indicators seemed to fulfil their purpose well.
Statoil is a large organization with a large number of different electronic information and management systems in use, many of which are seen as quite complex to navigate. Introducing an additional system in the form of early warning could easily be seen as further increase of the bureaucracy and render the measurements used less than desired. In the end, the measurements were integrated into an existing balanced scorecard-like system which avoided the introduction of a new module.
6 Conclusions and consequences
Experiences reported in this article show that the concept of early warning has potential, and that it can be operationalized through relatively simple tools. A Project Performance Measurement System must be easy and simple to use, must focus on important issues for the project and must give early indications or warnings about the status for the project. The development of a conceptual framework for early warnings in project is a work in progress. Further research on a project performance management system for early warning should focus on the following questions:
Which are the measurement areas to be included in an early warning system? What applies to all projects and what is project or industry specific?
Will existing data suffice or will the concept require additional data gathering? Can the same approach be used throughout the project, or is it phase specific?
7 References
Baker, N., D. Murphy, and D. Fisher (1983) Project Management Handbook: Factors Affecting Project Success, Van Nostrand Reinold, New York.
Cleland, D. I. (1994) Project Management, Strategic Design and Implementation, McGraw-Hill Inc.
Flyvbjerg, B., N. Bruzelius and W. Rothengatter (2003), Megaprojects and Risks: An Anatomy of Ambition. Cambridge : Cambridge University Press.
Fortune, J. and D. White (2006): Framing of project critical success factors by a systems model. International Journal of Project Management. 24, pp 53-65.
Harrison, E.F (1992), Some factors involved in determining strategic decision success, Journal of General Management, pp.72-87.
Kerzner, H. (1987) In Search of Excellence in Project Management. Journal of Systems Management, 38, 30. Kerzner, H. (1995) Project Management. A Systems Approach to Planning, Scheduling and Controlling, fifth edition, Van Nostrand Reinhold Co, USA.
Lewis, J. P. (1993) The Project Manager's Desk Reference, Probus Publishing Com. USA.
Lientz, B. P. & K. P. Rea, (1995) Project Management for the 21st Century, San Diego, Academic Press. Lim, B. T. C. (1987) Causal Modeling Construction Project Performance. Edinburgh, Heriot-Watt University. Merrow, E.W. (1998): Understanding the Outcomes of Megaprojects, A Quantitative Analysis of Very Large Civilian Projects, March, Rand Report.
Miller R., and D. Lessard (2000), The Strategic Management of Large Engineering Projects: Shaping Institutionss, Risks, and Governance, Massachusetts Institute of Technology, USA.
Nikander, I. O. & E. Eloranta, (1997) Preliminary signals and early warnings in industrial investment projects. International Journal of Project Management, 15, 371.
Nikander, I. O. & Eloranta, E. (2001) Project management by early warnings. International Journal of Project Management, 19, 385.
Nikander, I. (2002) Early Warnings: A Phenomenon in Project Management, Dissertation for Doctor of Science in Technology, Helsinki University of technology, Espoo, Finland.
Nikander, I. (2002): Early Warnings: A Phenomenon in Project Management, Dissertation for Doctor of Science in Technology, Helsinki University of Technology, Espoo, Finland.
Pinto, J.K., J.E. Prescott, (1988). Variations in critical success factors over the stages in the project life cycle. Journal of management, 14, 5-18.
Pinto, J. and D.P. Slevin, (1987) Critical Factors in Successful Project Implementation, IEE Transactions on Engineering Management, EM-34.
Pinto, J. and D.P. Slevin, (1988) Critical Success Factors across the Project Life Cycle, Project Management Journal, Vol. 19, No. 3, pp. 67-75.
Pinto, J. and O.P. Kharbanda, (1995) Successful Project Managers: Leading your Team to Success, Van Nostrand Reinold, New York.
Torp, O., O.M. Magnussen, N. Olsson, & O.J. Klakegg, (2006) Kostnadsusikkerhet i store statlige
investeringsprosjekter. (Cost uncertainty in large public investment projects) Concept-report no 15. the Concept programme, Trondheim, Norway.
