Performance Assessment

Performance Assessment

2012 Edition

Project Cost Growth

Owner (N

=975)

Project Schedule Growth

NPV = $ 6.4 Billion

NPV = $ 5.7 Billion

NPV = $ 6.6 Billion

NPV = $ 6.8 Billion

NPV = $ 7.7 Billion

“It isn’t what we don’t know that gives us trouble, it’s what we know that ain’t so.”

For the past 17 years, CII’s Performance Assessment Program has helped many CII members understand, and improve, the performance of their projects. Through this research program, CII has been able to quantify what works and what does not work when managing projects. Yet, this knowledge has largely not improved the business benefits received from the capital projects themselves. Why?

This issue of Performance Assessment examines the economics and realities of project execution in light of their ability to deliver value. Its findings present both challenges and opportunities for the field of project management going forward. But, ultimately, this study should encourage a wider discussion about project benefits amongst executives, project sponsors, and project teams (i.e., owners, allied E/C contractors, vendors, and suppliers).

Macroeconomics

In the early 1990’s, 90% of observed variation in an ‘average’ CII Owner’s cash flow (e.g., Cash Flow from Operating Activities (CFfOA)) was attributable to the delivery of capital projects (e.g, Construction in Progress (CIP)). At the time, the benefits of executing capital projects were evident. However, by 2008-2010, this figure had eroded to the point where only 20% of additional cash flows could be explained by the presence of new capital projects.

Following a couple significant recessions, CII hypothesizes that several factors may be at work lessening the impact of capital projects on cash flow. One is the presence of increased merger and acquisition (M&A) activity through the last decade – if additional cash flows could be purchased at a discount relative to executing capital projects, less projects would be authorized. A second factor could be the increased presence of regulation or maintenance wherein required projects generate little or no benefit. A third factor might be that the cost of capital projects has outpaced the cash flows available from them. Finally, CII owners are observed to exhibit higher levels of risk aversion in their capital projects following a recession (this phenomenon can be seen in the figure above where the ratio of cash flow to construction spending increases each year following a recession).

CII Owners’ Capital Efficiency

(Ratio of Cash Flow (CFfOA) to Construction In Progress (CIP))

2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 CA fO A/CIP Year 2.53 (2012) 3.68 (1999) N=64 Early 1990s

“The secret of business is to know something that nobody else knows.”

Using benchmarking data provided to CII by its owner members, along with financial data from The University of Texas at Austin’s McCombs School of Business, CII staff were able to create a cash flow diagram (see below) for an ‘average’ owner project. This ‘average’ project costs $65 million and has an internal rate of return (IRR) of 14.1% over a 20 year planning horizon. This result is in line with the IRR calculated by Morningstar (14.7%) for U.S.-based oil and gas companies and is consistent with the ‘hurdle rate’ used by CII owners during capital budgeting.

Unfortunately, it is at this point (i.e., capital budgeting) where the erosion of project benefits begins. Oftentimes, the anticipated financial returns from a capital project turn out to be optimistic. This may arise from a poor understanding of the product’s market, optimistic estimates of the cost and schedule of the project itself, or both. CII research has conclusively shown that full funding authorization should not be awarded when a project has poor scope definition or inadequate business planning as this leads to poor project performance. However, what was not understood (until now) is that the predictability of simultaneous cost and schedule performance is crucial for the attainment of financial performance stemming from a company’s portfolio of projects.

“Whenever an individual or a business decides that success has been attained, progress stops.”

industry has improperly decided that it is acceptable to pay more for schedule or to take excess time in the interest of saving money. The problem isn’t so much that project managers are optimizing cost or schedule, but rather, that in doing so, they are seriously compromising the company’s financial performance. A recent review of 975 owner-submitted projects in CII’s Performance Assessment Database revealed that only 5.4% of these projects met both their authorized cost and schedule within an acceptable margin (i.e., projects exhibiting first quartile delta cost growth and first quartile delta schedule growth). Worse, nearly 70% of the submitted projects had actual costs or schedules exceeding +/- 10% deviation from their authorized values (statistics shown on the next page).

– ARistotle onAssis – thoMAs J. WAtson -$5 M -$20 M -$40 M $9.5 M $14.4 M -$50 -$40 -$30 -$20 -$10 $0 $10 $20 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 Cas h Fl ow ($ M ill io n) Year “Average” CII Owner Project = $65 Million

Slope of Revenue = 2.7% per year (Incremental Rate of Corporate CFfOA)

IRR = 14.1%

Startup

CII Calculated Hurdle Rate for Industrial Projects

“Average” CII Owner Projected Cash Flow

(Morningstar Estimated Hurdle Rate for Oil and Gas Companies = 14.7%)

In the figure, five (5) regions are shown. Four of the regions are divided by positive or negative cost and schedule growth relative to 0%, respectively. The region containing projects with both first quartile delta cost growth and delta schedule growth is shown in pink (i.e., projects with ‘best in class’ predictability). Each region is listed with its corresponding percentage of the 975 owner-submitted projects and its average cost and schedule growth. For reference, the red region contains projects whose actual costs and schedules exceeded their budgeted values.

“Effective leadership is putting first things first. Effective management is discipline,

carrying it out.”

More than anything, an owner’s executive leadership team needs predictable project performance. Note that this does not imply that the projects have to be overly cost effective relative to projects executed by other companies; that is of secondary importance. No, projects must be predictable in both cost and schedule.

The bulk of today’s capital projects are poorly managed. Most companies (e.g., owners, contractors, and the extended supply chain) do not possess management processes that are in control (statistically or otherwise). The tragedy is that project execution does not have to be left to chance. A primary way to ‘shore up’ deficient project planning and execution processes is through the implementation of known CII practices, best practices, and tools. CII can even provide training and educational resources to assist in this pursuit. Eventually though, each company has to use this research and knowledge to find its own ways to predictably plan and execute capital projects.

The benefit of predictable project performance is best seen in the aggregate. Using data obtained for 64 publically-traded CII owner companies (courtesy of the Capital IQ database at the McCombs School of Business (University of Texas)), a hypothetical ‘average’ CII owner company was derived. In the figure on the next page, the Cash Flow from Operating Activities (CFfOA) is shown relative to this firm’s Construction In Progress (CIP) spending.

– stephen Covey

Project Cost Growth

Owner (N

=975)

Project Schedule Growth N=310 (31.8%)

Avg. Cost Growth = -10.2% Avg. Schedule Growth = 29.1%

N=239 (24.5%) Avg. Cost Growth = 16.2% Avg. Schedule Growth = 32.0%

N=102 (10.5%) Avg. Cost Growth = 12.3% Avg. Schedule Growth = -9.8% N=271 (27.8%)

Avg. Cost Growth = -12.7% Avg. Schedule Growth = - 8.2%

N=53 (5.4%)

Avg. Cost Growth = - 0.47% Avg. Schedule Growth = 0.24%

Sixteen years of data are shown (1996-2011) in the figure. These were subsequently used to create projections of cash flow and construction for the next five years (2012-2016). Interestingly, while cash flow increases at a constant rate, construction spending is forecasted to increase exponentially.

One primary assumption of the ‘average’ CII owner company forecast is that it will be able to predictably plan and execute capital projects (i.e., at ‘best in class’ levels (pink region performance)). But what if it does not? Shown above are two figures, each containing a set of ‘as is’ and ‘to be’ cash flow diagrams. The figure on the left shows the impact of ‘red region’ cost and schedule overruns. Note that the cash flows are both higher and spaced out to the right. Similarly, the figure on the right shows the impact of ‘blue region’ cost and schedule savings relative to budget.

Predicted CIP Predicted CFfOA UCL_CIP LCL_CIP LCL_CFfOA UCL_CFfOA -$4,000 -$3,000 -$2,000 -$1,000 $0 $1,000 $2,000 $3,000 $4,000 $5,000 $6,000 $7,000 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Ca sh F lo w ($ M ill io n) Year

CIP CFfOA Predicted_CIP Predicted_CFfOA

$ 4.107 Billion (2012)

-$ 2.165 Billion (2012) Source: Capital IQ Courtesy of McCombs School of Business, UT Austin

Cost Growth = -12.7% Schedule Growth = -8.2% -$4,000 -$3,000 -$2,000 -$1,000 $0 $1,000 $2,000 $3,000 $4,000 $5,000 2011 2012 2013 2014 2015 2016 2017 2018 Ca sh Flow ($ M ill ion) Year $3,507 M $3,627 M $3,747 M $3,867 M $3,987 M -$1,890 M -$1,980 M -$2,071 M -$2,162 M -$2,251 M

NPVTarget= $7.6 Billion NPVScenario 2= $6.8 Billion 11.1% Loss of NPV

To-Be Cash Flow

-$4,000 -$3,000 -$2,000 -$1,000 $0 $1,000 $2,000 $3,000 $4,000 $5,000 2011 2012 2013 2014 2015 2016 2017 2018 Cas h Fl ow ($ Mi lli on) Year $4,017 M $4,155 M $4,292 M $4,430 M $4,567 M -$2,165 M -$2,268 M -$2,372 M -$2,476 M -$2,579 M

As-Is Cash Flow

Cost Growth = 16.2% Schedule Growth = 32.0% -$4,000 -$3,000 -$2,000 -$1,000 $0 $1,000 $2,000 $3,000 $4,000 $5,000 2011 2012 2013 2014 2015 2016 2017 2018 Cas h Fl ow ($ Mi lli on) Year $4,017 M $4,155 M $4,292 M $4,430 M $4,567 M -$2,165 M -$2,268 M -$2,372 M -$2,476 M -$2,579 M -$4,000 -$3,000 -$2,000 -$1,000 $0 $1,000 $2,000 $3,000 $4,000 $5,000 2011 2012 2013 2014 2015 2016 2017 2018 Ca sh Flow ($ M ill ion) Year $4,017 M $4,155 M $4,292 M $4,430 M $4,567 M -$2,516 M -$2,635 M -$2,756 M _{-$2,877 M -$2,997 M}

NPVTarget= $7.6 Billion NPVScenario 1= $5.7 Billion 25.3% Loss of NPV

As-Is Cash Flow

“If a business does well, the stock eventually follows.”

Crawl, walk, and run – in that order. Cost effective projects are possible, but should not be achieved ‘by chance.’ As a discipline, project management needs to ‘walk’ before trying to ‘run.’ Companies (e.g., owners, contractors, vendors, and suppliers) must work together to develop efficient processes and effective organizations. Doing so will give these companies the foundation and capabilities needed to be more predictable and outperform their peers. Fortunately, CII has uncovered several interesting findings related to preserving and advancing the corporate benefits of predictable capital projects. The top four findings can be seen in the table below.

The ‘average’ CII owner can expect to achieve benefits of $6.45 Billion in NPV (five (5) year planning horizon) from their portfolio of capital projects given existing levels of project predictability. This benefit falls well short of the established target(s) for this portfolio at $7.65 Billion in NPV. The inability to predictably deliver both cost and schedule results in the loss of 16% of NPV, or, $1.2 Billion.

Through the CII Performance Assessment Database, the research results indicate that owners could gain $500 Million of NPV through the combined application of: 1) using a lump sum contract, 2) working with a CII contractor, 3) achieving top PDRI scores, and 4) implementing the CII best practice of Planning for Startup. As good as this may sound, not doing these things loses over $2.1 Billion in NPV.

It is important to remember that these findings result from real, submitted projects in CII’s database. Here, the track record of CII owners executing cost reimbursable work is not good. This is one reason why CII recently conducted research into the best ways to execute cost reimbursable work. In addition, it is thought that many of the cost reimbursable projects had poor scope definition as evidenced by high PDRI scores at authorization. Going forward, CII companies must work together to achieve higher levels of project predictability. The research is clear: predictable projects produce good financial results.

Although counterintuitive, negative cost and schedule growth projects actually constrain capital investment. That is to say, other Net Present Value (NPV)-producing projects could have been underway if the existing project had not ‘tied up’ capital for one or more years. A net 25.3% loss of NPV is observed for the positive cost and schedule growth projects. A net 11.1% loss of NPV is observed for the negative cost and schedule growth projects. These findings should be alarming. They should also be a call to action.

– WARRen Buffett

Practices Expected NPV Gain/Loss Improvement

CII Owners’ Average $ 6.45 Billion N/A N/A

Contract Method Lump Sum $ 6.81 Billion $ 360 Million 5.5% Cost Reimbursable $ 5.50 Billion - $ 950 Million -14.8% Working Relationship Work w/ CII Contractor $ 6.80 Billion $ 350 Million 5.3% Work w/ Non-CII Contractor $ 4.61 Billion - $ 1,840 Million -28.5%

PDRI <=200 $ 6.48 Billion $30 Million 0.5%

>200 $ 6.10 Billion - $360 Million -5.6% Planning for Startup High Use $ 6.45 Billion $ 0 Million 0.0% Low Use $ 6.23 Billion - $220 Million -3.4%

“High expectations are the key to everything.”

Achieving optimal levels of financial returns from capital projects is CII’s mission. While a lot has been accomplished, much more remains to be done. CII will continue to ‘push the envelope’ through its Performance Assessment and Re-search programs to find ways to achieve optimal business results. The path forward will not be easy and will demand the energy, innovation, and commitment from those involved.

It has been said that the journey of a thousand miles begins with a single step. Today, CII members have tremendous resources that they can use to achieve higher levels of predictability in their project work. One of the best CII resourc-es remains its Performance Assresourc-essment Program. Being able to objectively evaluate a project relative to similar proj-ects submitted by other companies is a tremendous benefit for a CII member. Knowledge of this type drives action; to research, to train, or to implement solutions which will improve project execution and increase corporate financial performance.

– sAM WAlton

A clear opportunity exists for CII members to drive business performance through capital projects. The good news is that CII has identified practices and tools known to increase corporate financial returns. The bad news is that the Engineering and Construction (E/C) Industry has a long way to go. In fact, CII has determined that if every project in the ‘average’ CII owner’s portfolio could achieve optimal levels of cost and schedule growth, the resulting NPV would approach $8.00 Billion. This figure represents a 4.6% increase over the existing NPV target. Importantly, optimal predictability would increase NPV by 24.0% (when compared to the expected financial returns of today’s capital projects).

Target NPV

Expected NPV

B.P. Enhanced NPV

Optimal NPV

$7.65 B

$6.45 B

$6.95 B

$8.00 B

KNOWN

UNKNOWN

(RESEARCH)

Cii performance Assessment today: By the numbers

The CII Performance Assessment Program is a user-friendly, resource efficient, statistically credible benchmarking system. As a non-profit, university-based research institute, CII is uniquely positioned to provide quantitative project performance information to its member companies. Since 1996, the program has accomplished the following:

• assessed 2,039 projects worth over $133 Billion • trained 1,070 people as Benchmarking Associates • involved 140 companies submitting at least one project • produced over 45 Benchmarking reports and publications • been advised by 155 current and former committee members.

Today, the program employs 10 staff members to advance project performance through benchmarking research. CII’s Performance Assessment staff also produces publications loaded with analyses of industry performance and trends. The benefits of analyzing projects with CII are numerous and all CII Member companies can submit projects for free. The starting point is CII’s online Performance Assessment System (PAS), where real-time project performance data can be obtained.

To become involved in CII’s Performance Assessment Program, please contact:

stephen p. Mulva, ph.D.

Jiukun (Jason) Dai, ph.D.

Associate Director

Research engineer

e-mail: [email protected]

e-mail: [email protected]

phone: (512) 232-3013

phone: (512) 232-3050

http://www.construction-institute.org/benchmarking

The Knowledge Leader for Project Success

Owners • Contractors • Academics

© 2012 Construction Industry Institute™ The University of Texas at Austin

CII members may reproduce and distribute this work internally in any medium at no cost to internal recipients. CII members are permitted to revise and adapt this work for their internal use, provided an informational copy is furnished to CII.

Available to non-members by purchase; however, no copies may be made or distributed and no modifications may be made without prior written permission from CII. Contact CII at http:// construction-institute.org/catalog.htm to purchase copies. Volume discounts may be available. All CII members, current students, and faculty at a college or university are eligible to purchase CII products at member prices. Faculty and students at a college or university may reproduce and distribute this work without modification for educational use.