DEVELOPING A MODEL TO EVALUATE PROJECT
PERFORMANCE: CONTRACTOR COMPANY’S VIEWPOINT
Mohammad Ilbeigi
M.Sc. student, School of Civil Engineering, University of Tehran, Tehran, Iran [email protected]
Gholamreza Heravi
Assistant Professor, School of Civil Engineering, University of Tehran, Tehran, Iran [email protected]
Project success is one of the most important concepts and goals in project management, for which it seems to be difficult to give an exact definition. In this paper, the authors used a comprehensive definition of project success consisting of two components: product success and project management success. Due to the insufficient attention given to the study of project management performance in contracting companies, while merely considering the consequences of the final product, in this paper the level of project management performance from a contracting company’s point of view was evaluated by developing a novel model. Critical and effective indices were defined for this component and used to develop a model, by which the project management performance can be evaluated.
KEYWORDS: project management performance, critical performance indices, contractor company.
INTRODUCTION
A review of the project management literature provides no consistent interpretation of the term “project success”. McCoy (1986) observed that a standardized definition of project success does not exist nor does an accepted methodology of measuring it. Likewise Wateridge (1998) noted that very few people in the past have thought seriously about the success criteria. Similarly, Wells (1998) laments the lack of attention given to defining success except in quite general terms. Furthermore, one of the most important efforts for quantifying the project performance is “The Practice Standard for Earned Value Management” (EVM) which has been developed as a supplement to “A Guide to the Project Management Body of Knowledge” (PMBOK). This standard focuses only on cost and schedule performance. Moreover, Kamil Nassar (2005) developed a model to evaluate project performance. His model consists of 8 indices which try to evaluate project performance but this model does not separate indices considering the difference between product success and project management success and does not focus on the project processes.
Also, since the definition of project success is directly related to the stakeholders’ various goals and benefits, which are sometimes in contrast with each other, the first step in defining project success is identifying whose point of view should be considered. In this paper, project performance based on the point of view of the contractor and project executing company is studied.
Baccarini (1999) emphasized that project success consists of two components, as follows:
• Product success deals with the effect of the project’s final product.
• Project management success focuses upon the project process and, in particular, the successful accomplishment of cost, time, and quality objectives. It also considers the manner in which the project management process was conducted.
Nevertheless the long-term success of a contractor company depends on its performance in project management, lots of contractors consider only product success and its consequences. Consequently many of the long-term goals of a contractor organization and also its condition in competitive markets are endangered. Obviously if a contractor repeats its mistake in other projects, this means it has a serious problem in its project management system, even though it may have many successful projects products.
Considering the lack of a mathematical and quantitative model for measuring the project management performance based on the Baccarini’s definition, focusing on the project processes, a need to develop such a model exists. Thus in this paper, by developing a novel model, the level of project management performance from a contractor company’s point of view is evaluated. Moreover some critical and effective indices are defined which, by calculating and substituting them in the developed evaluation performance function, the value of performance is measured. Furthermore, sub-indices of performance will be introduced which can specify the strong and weak points in project management success.
PROJECT MANAGEMENT INDICES
As mentioned above, project management performance indices concerned with the successful accomplishment of the project’s cost, time, and quality objectives and consist of the six following indices:
Cost Performance Index (CPI)
The Cost Performance Index (CPI) is a measure of the cost efficiency of the project. The CPI is determined by dividing the earned value by the actual costs incurred. This index is one of the indices of “The Practice Standard for Earned Value Management” (EVM).
CPI = (BCWP) / (ACWP)
Where BCWP is the Budgeted Cost of Work Performed and ACWP is the Actual Cost of Work Performed.
The category limits of the performance for each index is determined by the contractor organization’s manager’s expert opinion of contractor organization, regarding the index definition, contractor’s strategies, market rivals, etc. These category limits can be modified for every project. The limit values determined in this paper are only for the present case study. Table 1 shows the limits for the CPI index.
Table 1: Cost Performance Rating Index Range Rating Condition CPI ≥ 1.15 Outstanding Performance 1 1.05 ≤ CPI < 1.15 Exceeds Target 2 0.95 ≤ CPI < 1.05 Within Target 3 0.85 ≤ CPI < 0.95 Below Target 4 CPI < 0.85 Poor Performance 5
Billing Performance Index (BPI)
The billing or cash flow performance index shows the contracting company’s ability and performance in receiving claims and demands and it is defined as follows:
BPI = (CRWP)/(PRWP)
Where PRWP is the Potential Revenue of Work Performed and CRWP is the Cash Revenue of Work Performed.
Table 2: Billing Performance Rating
Index Range Rating Condition BPI ≥ 0.98 Outstanding Performance 1 0.95 ≤ BPI < 0.98 Exceeds Target 2 0.90 ≤ BPI < 0.95 Within Target 3 0.80 ≤ BPI < 0.90 Below Target 4 BPI < 0.80 Poor Performance 5
If the BPI index indicates an undesirable situation, the contractor’s lack of financial security may be the result inappropriate cord codifying by the contractor or the client’s neglect of his financial commitment to the project.
This issue is considers by the BPIclient and BPIcontractor sub-indices. The former considers the
client’s influence and the latter considers the contractor’s influence in financial issues of the project.
BPIContractor = (BRWP)/(PRWP)
Where BRWP is the Billed Revenue of Work Performed.
Schedule Performance Index (SPI)
The Schedule Performance Index (SPI) is a measure of the schedule efficiency of the project. Similar to the Cost Performance Index (CPI), SPI is one of the indices of “The Practice Standard for Earned Value Management” (EVM).
SPI = (BCWP)/(BCWS)
Where BCWP is the Budgeted Cost of Work Performed and BCWS is the Budgeted Cost of Work Scheduled.
Table 3: Schedule Performance Rating
Index Range Rating Condition SPI ≥ 1.15 Outstanding Performance 1 1.05 ≤ SPI < 1.15 Exceeds Target 2 0.95 ≤ SPI < 1.05 Within Target 3 0.85 ≤ SPI < 0.95 Below Target 4 SPI < 0.85 Poor Performance 5
Safety Performance Index (SFI)
The index considers issues about safety observation and increasing safety in executive project outcome. The SFI index is the sum of two weighted sub-indices as follows:
SFIC = (ELSO)/(TECP)
Where SFIC is the safety performance index of the project outcome in terms of expenses, ELSO
is the Expenses arise from damages of the lack of safety observance and TECP is the Total expenses of project construct phases
SFIS = (IDWH)/(TIWH)
Where SFIS is the safety performance index of the project outcome in terms of scheduling,
IDWH is the Number of individual - dismissed working hours because of lack of safety observance and TIWH is the Total number of individuals–working hours in construction phase. Obviously the SFIC and SFIS must be normalized before calculation of SFI, so:
SFI = x1×SFIc + x2×SFIs
Σ xi = 1
Where Xi is the weighting factor of sub-Indices
Table 4: Safety Performance Rating
Index Range Rating Condition SFI ≥ 1.15 Outstanding Performance 1 1.05 ≤ SFI < 1.15 Exceeds Target 2 0.95 ≤ SFI < 1.05 Within Target 3 0.85 ≤ SFI < 0.95 Below Target 4 SFI < 0.85 Poor Performance 5
Quality Performance Index (QPI)
This index considers the quality of the project outcome based on the expenses resulting from the lack of quality, the expenses of redoing part of the work, fines, etc.
QPI = (SELQ)/(TECP)
Where SELQ is the sums of direct and indirect expenses arise from lack of quality in project outcome and TECP is the total expenses of project construct phases.
Table 5: Quality Performance Rating
Index Range Rating Condition QPI ≤ 0.005 Outstanding Performance 1 0.005 < QPI ≤ 0.01 Exceeds Target 2 0.01 < QPI ≤ 0.02 Within Target 3 0.02< QPI ≤ 0.03 Below Target 4 QPI > 0.04 Poor Performance 5
Environment Performance Index (EPI)
This index considers the contractor’s performance regarding environmental issues and expenses arising from the lack of observance of environment consideration during the project.
EPI = (SELE)/(TECP)
Where SELE is the sum of the direct and indirect expenses arising from the lack of attention to the environmental issues and TECP is the total expenses of project construct phases.
Table 6: Environment Performance Rating
Index Range Rating Condition EPI ≤ 0.002 Outstanding Performance 1 0.002 < EPI ≤ 0.005 Exceeds Target 2 0.005 < EPI ≤ 0.01 Within Target 3 0.01 < EPI ≤ 0.015 Below Target 4 EPI > 0.015 Poor Performance 5
PROJECT MANAGEMENT PERFORMANCE MODEL
The function of project management which considers the manner of management of the project’s outcome is defined as follows, by combining the six critical performance indices:
PMPI= u1×CPI+u2×BPI+u3×SPI+u4×SFI+u5×QPI+u6×EPI
Σ ui = 1
Where ui is the weighting factor for each of the critical indices of project management success.
It is necessary to mention that weighting factors for each of the indices are determined by AHP and consistency measurement in order to make sure of the reasonability of the results. Moreover the values of the critical index and sub-index must be normalized in order to be used in the project management performance function.
Normalization of Performance Indices
As the first step in normalization of the indices, the success borders for the CPI index as the base borders are determined. On the basis of the base borders and by using the ratio of the success border of each index and the basis success borders (or by using a diagram as shown in figure 1),
the normal amount of each performance index is calculated. As an example, the diagram for calculating the normal amount of BPI according to the CPI category limits, is shown in Figure 1.
Figure 1: The Typical Diagram for Calculating Normal Amount of BPI According to the Limits of CPI.
CASE STUDY
In this section, the developed performance evaluation model will be applied for the case of a real project which is being executed in Iran. The scope of the project is the construction of a power transmission line (consisting of 176 telescopic towers), 22 Km long.
By studying the monthly documents of this project which were recorded at the end of the 4th month (January, 2009) of the project’s timetable, the following information was extracted:
• Budgeted Cost of Work Performed (BCWP): 2,600,000$
• Actual Cost of Work Performed (ACWP): 3,100,000$
• Budgeted Cost of Work Scheduled (BCWS): 2,550,000$
• Cash Revenue of Work Performed (CRWP): 2,822,000$
• Billed Revenue of Work Performed (BRWP): 3,180,000$
• Potential Revenue of Work Performed (PRWP): 3,300,000$
• Earned Profit of Work Performed (EPWP): 525,000$
• Expenses arise from damages of the lack of security observation (ELSO): 0 $
• Total expenses of project construct phases (TECP): 4,800,000$
• Number of individual - dismissed working hours because of lack of security observation (IDWH): 1920 Individual-Hour (The project had been placed on hold for 8 days because of safety problems)
• Total number of individuals - working hours in construction phase (TIWH): about 72,000 Individual-Hour
• The sums of direct and indirect expenses arise from lack of quality in project outcome (SELQ): 12,000$
• The sums of direct and indirect expenses arise from lack of attention to the environmental issues (SELE): 0$
Using the above information, the critical indices and their normal amounts ware calculated and the results presented in table 7.
Table 7: The Amount of Critical Performance Indices for the Power Transmission Line
Index Amount of Index Condition Normal Amount
CPI 0.84 5 0.84 BPI 0.85 4 0.90 SPI 1.02 3 1.02 SFI 1.12 2 1.12 QPI 0.002 1 1.19 EPI 0 1 1.15
The BPI sub-indices are calculated as follows: BPIClient = 0.89 and BPIContractor = 0.96
And the calculated SFI sub-indices are as follows:
SFIC = 0.00 (Normal value = 1.15) and SFIS = 0.03 (Normal value = 1.05)
Where, the weighting factors of the SFIC and SFIS are 0.7 and 0.3, respectively.
Project Management Performance Evaluation
As mentioned in the previous section, the weighting factors of critical performance indices are determined by AHP. As a result the weighting factors for this project are shown in table 8 (the consistency is 0.063, so the weight factors are reasonable and acceptable).
Table 8: The Weighting Factors of the Critical Performance Indices for the Power Transmission Line
Index CPI BPI SPI SFI QPI EPI
Weighting Factor 0.26 0.20 0.19 0.13 0.15 0.07
On the basis of the above weighting factors, the amount of the project management performance function for this project is derived as follows:
PMPI= 0.26×0.84+0.20×0.90+0.19×1.02+0.13×1.12+0.15×1.19+0.07×1.15 PMPI= 0.997
Finally, the value of the project management performance of this project is determined to be 0.997. Regarding the category limits of the base border, the performance condition of this project is within the target range (condition 3).
Analysis of the Results
The results of this case study show that the weakest performance of the contractor is the cost performance which is in the 5th condition (poor performance). This poor cost performance may be the consequence of a mistake in cost forecasting before the beginning of the project or the poor cost performance of the contractor along with the project. For a more precise evaluation, CPI can be divided into the sub-indices such as Indirect Cost Performance Index (CPII), Labor
Cost Performance Index (CPIL), Material Cost Performance Index (CPIM).
The best performance of the contractor is QPI, which is in the 1st condition (outstanding performance) showing that the contractor has an excellent performance quality. The EPI index is also in the 1st condition, but because of its weak weighting factor it is not very effective on the project management performance of contractor.
Sensitivity Analysis
The project management performance can change as the value each index changes. If the contractor wants to conduct corrective actions to improve its project management performance, it is necessary to recognize the most important indices. Figure 2 shows the change in the project management performance due to changes of each index as the other indices remain unchanged. Sensitivity analysis shows that the CPI, having the maximum slope, is the most effective index, and the EPI which has the minimum slope has the least effect on project performance.
Figure 2: T he Sensitivity A nalysis Diagram o f the Project Man agement Performance in Respect of the Normal Amount of Indices
0,9 0,92 0,94 0,96 0,98 1 1,02 1,04 0,7 0,8 0,9 1 1,1 1,2 1,3 Pr oje ct M an ag em en t P er for m an ce
Normal Amounts of indices
CPI BPI SPI SFI QPI EPI
CONCLUSIONS
In this paper, project management performance was studied based on an index-based model. For the project management performance, the six critical performance indices consisting of: Cost Performance Index, Billing Performance Index, Schedule Performance Index, Safety Performance Index, Quality Performance Index, and Environment Performance Index, were defined. As mentioned before, due to the conflict of interest between the contractor and the other stakeholders, the point of view of the contractor was considered in this paper. Thus all of the indices were measured by considering the profits and damages of the contractor. It is necessary to mention that Billing Performance Index is only defined for contractors. Furthermore the function for project management performance which considers the manner of management of the project’s outcome in terms of introduced critical performance indices was defined. Finally, by application of the developed model for a real project, performance indices, weighting factors and project management performance of a contractor company were evaluated and by analyzing the results of the model, the weak and strong points of the contractor’s project management were specified. The results show that this model can measure project performance with suitable accuracy and can be used as a reliable tool for measuring the project management performance of contractors.
REFERENCES
Baccarini, D. (1999) The Logical Framework Method for Defining Project Success, Project Management Journal; Dec 1999; 30, 4; ABI/INFORM Global pg. 25
Nassar, N.K., An integrated framework for evaluation, forecasting, optimization of construction projects, The thesis for PhD degree, University of Alberta, 2005.
McCoy, F.A. (1986). Measuring success: Establishing and maintaining a baseline, PMI Annual Seminar & Symposium Montreal.
PMI (Project Management Institute), (2005). Practice Standard for Earned Value Management Wateridge, J. (1998). How can IT/IS projects be measured for success? International Journal of Project Management Journal, 16(1), 59-63.
