Stage A – Search Procedures

This stage entails the assessment of existing body of literature available and the

39

industrial practices which will serve as the study`s reference point. Manual and computer searches was conducted extensively on the subject matter with reference to construction and related literature. Two key important steps were involved in this stage: the first step was intended to acknowledge the current state of the AEC industry knowledge; this step results in the location of 28 relevant studies, while the second step was intended to categorize some of the important metrics needed to be analyzed in order to accomplish the research goal; this step results in the creation of table 2. According to the literatures, the project delivery systems performance metrics identified both qualitatively and quantitatively are the dependent variables that were measured upon project completion.

The project delivery systems` key performance, benefits, or the levels of waste metrics analyzed in this meta-analytic study was based primarily on the performance metrics identified within earlier research works and/or case studies. Some of these performance metrics are highlighted in the Literature Review section which includes journal papers, technical reports, and conference proceedings. These reviews thus resulted in the dataset that are presented in this study (see data analysis section). Table 2 below presents the list of the various studies and dataset gathered for the purpose of this study. An inclusion rule was generated for each individual study in order to obtain better and non-biased results. Each study was included if: (1) it conveyed sufficient information to allow for statistical data accumulation (2) a statistical description of the data was provided in study. These studies were carefully searched and included following an appropriate keyword search (e.g. project delivery methods, competitive sealed proposal, design-bid-build, delivery performance, project delivery and construction wastes, quantifying

40

construction waste, partnering in construction, project delivery method procurement, construction cost and time control, and construction schedule) into the internet search engine publishers such as: Elsevier, ASCE Library, Google Scholar, AIC, Scopus, Emerald Insight, Science Direct, Research Gate etc.

The first stage procedures resulted in the identification of 28 relevant studies. The comprehensive list of information regarding each study are summarized in Table 2. The table chronologically identifies each study author(s), the year of the publication, publication type, project type (i.e. the various kind of project(s) that was identified in the study), and the project/study location. Out of the 28 studies identified, one study could not be found (Rakesh et al. 2013) due to the restriction placed by the researcher. In addition, five out of the studies acquired during the search process failed to allow for adequate meta-analytic combination due to the studies insufficient information and hence were not included in the study for further analysis. The studies are: Hyun et al.

2008, Molenaar et al. 2009, Riley et al. 2005, and Water Design–Build Council 2009.

Unlike with most methodology, one of the most common problem with meta-analysis is the availability of unpublished studies and the relevancy of studies. This exclusion of non-published studies increases selection bias (walker et al. 2008). However, the remaining 23 studies were subjected to the next stage (coding procedures) and included in the study for further analysis.

41

ReferenceStudy Author(s)Publication YearPublication TypeProject TypeProject Location 1Bennett et al1996JournalGeneral/Public SectorUnited Kingdom 2Pocock 1996JournalPublic SectorUSA 3Konchar M. and Sanvido V.1998JournalIndustrialUSA 4Ibbs et al.2003JournalGeneral (CII)USA, Canada, Middle East, Latin America 5Riley et al.2005JournalMechanical ProjectUSA 6Riley et al.2005JournalGreen Building ProjectUSA 7Groppel et al.2005ReportEducationDallas 8Col Debella, D. and Ries, R.2006JournalEducationUSA 9FHWA2006ReportHighwayUSA 10Rojas and Kell2008JournalPublic SectorUSA 11Hyun et al.2008JournalMultifamily HousingUSA 12Water Design–Build Council2009JournalWater & WastewaterUSA 13Molenaar et al.2009JournalPublic and Private ProjectsUSA and District of Columbia 14Hale et al.2009JournalMilitary BuildingsUSA 15Rajan et al.2010JournalMultiple Projects (ENR) USA 16Korkmaz et al.2010JournalGeneralUSA 17Beville et al.2010JournalEducationTexas 18Culp2011JournalWater & WastewaterUSA 19Touran et al.2011JournalTransit ProjectsUSA 20Shrestha et al.2011JournalHighwayTexas 21Kulkarni et al.2012Conference proceedingsPublic SectorUSA 22Texas Legislative Council2012ReportPublic SectorUSA 23Neidert et al.2012JournalEducationTexas 24Reinisch et al.2012JournalEducationTexas 25El Asmar et al.2013JournalGeneralUSA 26Rakesh et al. 2013JournalEducationTexas 27Minchin et al.2013JournalHighwayFlorida 28Nikou Goftar et al.2014Conference proceedingsPublic SectorUSA

Table 2: Basic information of identified studies

42 3.2.2 Stage B – Extraction/Coding Procedures

Following search procedure, two steps are necessary for the extraction and coding of the identified studies. According to Alarcón 1993, Serpell 1995, performance and waste levels in the construction and manufacturing industry includes such activities as delays, delivery times, quality, costs, lack of safety, rework (project change), long distances, unnecessary transportation trips, inappropriate delivery selection or management of methods, product and process design, equipment and poor constructability. The process of eliminating wasteful activities from the industrial ways of doing things can help improve levels of profitability, enhance competitive advantage, as well as enhance the levels of performance (Horman and Kenley 2005). Using the above description of waste and level of performance, studies were coded under the performance and waste metrics outlined (see table 3 and 4) with further discussion in the following section. Due to varying terminologies and the way studies are reported across studies, some studies`

findings reclassification was needed. For instance, in a study conducted by Kulkarni et al. (2011), cost performance in terms of project change for CSP was reported to be both positive and negative, confirming that CSP contracts may result in loss as well as cost savings. In order words, indicating that such performance under the project delivery method is a waste will be absurd. Project delivery performance levels could be categorized as wasteful, crucial, efficient or ineffective. All these factors are dependent on the owner’s choice of project team selection or selecting the most appropriate delivery systems. Overall, with varying degrees of terminology and detail, the literatures

43

identified in this studies have adopted the above classification system to an extent. The final stage of this research study contributes further to the aforementioned two stages and it entails further analyzing the identified studies in order to differentiate the listed performance metrics as shown in table 3.

3.2.3 Stage C – Statistical Procedures

Quantitative and qualitative descriptive statistics from the various identified studies were collected, combined, and included in the meta-analysis. The descriptive statistical analysis consists of testing whether DBB, CSP, DB, or CMR leads to better performance when compared against one and another. However, analysis such as univariate, multivariate, and Anova t-test were performed on each of the identified study dataset to compare the four project delivery methods as shown in table 5 to 9, and to test whether a particular delivery method (e.g. DBB) performs better than the other alternative project delivery systems (CSP, DB, and CMR) or vice versa, based on each of the identified performance and waste metrics listed for the purpose of this study. The descriptive statistics collected provides an estimate of the statistical nature of the variables of interest under each performance and waste metrics. For clarity and comprehensiveness, detailed description of each study and statistical significance value are further explored for each individual analyzed metric to determine each delivery system`s performance against another and to further identify and analyze the results whether a particular delivery system results in a higher level of performance, benefit or waste.

44

Italics not included in studies for further analyses

Table 3: Studies information categorized by listed metrics StudiesYearCost GrowthTime/Schedule GrowthConstruction SpeedProject Change (change orders)

Quality (sustainability, turnover) Data Set Bennett et al1996——332DBBDB Pocock1996——209DBBDB Konchar M. and Sanvido V.1998—351DBBDBCMR Ibbs et al.2003——67DBBDBOTHERS Groppel et al.2005———66CSPDBB Riley et al.2005———?DBBDB Riley et al.2005————120DBBDB Col Debella, D. and Ries, R.2006—88DBBDBCMR FHWA2006——22DBBDB Hyun et al.2008—24DBBDB Rojas and Kell2008———297DBBCMR Hale et al.2009——77DBBDB Molenaar et al.2009 Water Design–Build Council2009 Beville et al.2010——238DBBCSPCMRDB Rajan et al.2010—400DBBCMR Culp2011——?DBBDBDBODBFO Reinisch et al.2011————55CSPCMR Shrestha et al.2011——22DBBDB Touran et al.2011———9DBBCMRDBDBOM Kulkarni et al.2012———30CSPCMR Neidert et al.2012——33CSPCMR Texas Legislative Council2012——59CSPCMRDB Korkmaz et al.2013————12DBBDBCMR Rakesh et al. 2013——?CSPCMR El Asmar et al.201335IPDOTHERS Minchin et al.2013———60DBBDB

Metrics/Line Items Studies Delivery Methods

45

Table 4: Sample level of waste and performance metrics (identified within studies) Line Items Design – Bid– Build

(DBB)

Competitive Sealed Proposal (CSP)

References

Cost growth Positive effects of cost and productivity changes were

The table above shows the list of the identified performance and waste metrics that are used for the purpose of this research study. These metrics were based on some of the

46

aforementioned metrics included in earlier researches that has been highlighted in the previous section of this research study. In addition, the table briefly highlight some key findings of the line items studied in this paper by comparing the least collaborative project delivery systems (DBB and CSP) with the more collaborative project delivery systems (DB and CMR). Research studies and results are summarized under each delivery methods with each reference.

3.3 Data Analysis

The dataset identified for the purpose of this study are shown in Table 3, they were further broken down and discussed in the order of the identified key performance metric;

namely cost growth, time-schedule growth, construction speed, project change, and quality, which will be used to determine the levels of performance and waste between the four project delivery system (table 5 to 9). There are abundance of studies, reports, articles and researches that has been conducted comparing all four project delivery system against one and another in areas such as cost, schedule, time, and quality. Some studies compared three or up to five different delivery systems under some of the aforementioned metric. However, limited literature exist in terms of benchmarking the less collaborative DBB and CSP against one another. Although, this could be as a result of their closeness in characteristics (see Table 1.) and procurement (see figure 2 and 4), or even the lack of knowledge and awareness of their differences, which is the main intent of this research study. Since it was very difficult to collect studies and data that

47

solely compared DBB and CSP or all four delivery systems (i.e. DBB, CSP, DB, and CMR) together as an entity, enough studies were identified and collected to empirically and meta-analytically analyze each project delivery method performance. Thus, for the purpose of this study, all four project delivery system`s performance are benchmarked against one and another or collectively in some instances to help determine the levels of waste, benefits or performance of all four delivery system. Unlike Table 3, Table 5 – 9 below further identifies the statistical procedure and methods that were used for each studies. It also shows if the studies report any statistical significant p-value as well as other major findings that are within each studies. These are presented as either percentage increase or decrease or as a plus or minus. The study’s findings and result are discussed in the Result and Discussion section that follows.

3.4 Delimitations and limitations 3.4.1 Delimitations

This research chooses to focus mainly on the metrics defined in the table above. Since the metric are some of the important underlying factors within the architecture engineering construction (AEC) industry, the author intended to create more awareness by mainly focusing on these areas. Most literature however, also focused on the metrics used in the paper. Likewise, there are total of five identified literatures in this study that are not included in the study analyses. One of them is due to an embargo placed on the study, while the other four are due to the way they are reported. The intent of this paper is to continue with an in-depth analysis of the current state of the construction industry in

48

order to provide a better background knowledge and awareness of the identified project delivery methods for future purpose. Other metrics or underlying factor that are applicable for further research includes labor factor, constructability, production rate, safety, risk, communication, recycling rate, overhead and profit etc..

3.4.2 Limitations

Since it was very difficult to collect studies and data that solely compared DBB and CSP or all four delivery systems (i.e. DBB, CSP, DB, and CMR) together as an entity, the studies identified in the research are mainly either DBB or CSP performance being compared with other alternate delivery methods (i.e. CMR, DB). Likewise, the type of projects that were procured with the four project delivery systems ranges from residential type projects to highway construction. Acquiring literature or dataset on a specific project type was equally difficult. Unlike DBB and other alternative delivery systems, CSP delivery method is not a commonly used procurement system in the private sector and in most states in the United States. They are however frequently used in most ISDs, universities and government sectors. The above factors limit the availability of literatures, datasets and significant studies to compare the four project delivery systems using the identified performance metrics. Nevertheless, for the purpose of this study, the four delivery system`s performance are benchmarked against one and another to help determine the levels of waste, benefits or performance of DBB, CSP, DB, and CMR.

49

4. RESULT AND DISCUSSION

4.1 Research Findings

The systematic analysis of identified data set will be describe in the order of the listed performance and waste metric chronologically with each metric; namely cost growth, time-schedule growth, construction speed, project change and quality, which will be used to determine the levels of performance and waste between the two project delivery system. Following each table, each categorized metric and findings will be further

50

Cost performance on each delivery system is accessed through project cost growth. The total cost of the project as reported after the project is finished against the budgeted cost.

By definition, the level of the project delivery performance or waste is quantified in terms of percentage by comparing the difference between the actual and total construction award costs to the total contract construction award costs.

Actual Total Cost − Total Contract Award Cost.

51

Bennett et al. (1996) reported a 13% increase in cost growth in DBB over DB in the general/public sector construction in a research that was conducted in the United Kingdom. A study by Pocock (1996) on 25 public sector projects showed 12.8% cost growth for DBB and 6.7% for DB, which shows a 5.7% less cost growth for DB.

Likewise, in the study conducted by Konchar and Sanvido (1998) showed a similar result in favor of DB projects of about 5.2% less cost growth in DB usage as the project delivery system in comparison with DBB. In a study conducted by Ibbs et al. (2003) on 67 building projects using data from the construction industry institute (CII) showed a favorable 7.8% less cost growth for DBB. In order words a cost savings of 7.8% when compared with the DB projects. Groppel et al. (2005) reported a tremendous savings by the Dallas ISDs from switching from DB and CMR to CSP. A cost comparison of

$13.3M cost saving for CSP and $254.3 for both DB and CMR within the ISDs first quarter the same bond programme in 2004-2005. Col Debella and Ries (2006) studied a total of 88 educational project across the United States, all of which used DBB, CMR, and DB delivery system. The study showed a p-value of 0.557 with a 95% confidence level. In order words, this means there are non-statistical significant result in the three delivery system in their cost growth, they all performed well and within budget. In a report published by FHWA in 2006, 11 pairs of highway projects was investigated in different states in U.S. using DB and DBB delivery method. Favorably, DBB reflected a 3% less cost growth as compared to DB. Rojas and Kell (2008) looked at a total of 222 DBB project compared to 6 CMR in Washington and Oregon and found a highly statistically significant of 3.25% cost growth with DBB projects, while CMR projects

52

showed a 19.40% cost growth. In a study conducted by Hale et al. (2009) on 38 DBB and 38 DB naval building project showed only a 2% cost growth in DBB compared to DB. Beville et al. (2010) studied four delivery methods (DBB, CSP, CMR, and DB) on educational projects, the results showed a less cost growth in CSP and CMR projects compared to CSP and DB. A study analyzing ENR best projects in Texas by Rajan et al.

(2010) compared the impact DBB, CSP, CMR, and DB delivery methods on the various projects analyzed in the study showed a 2.5% cost growth in CSP projects as compared to a 7.69% cost growth in DBB projects. Reinisch et al. (2011) compared cost on public elementary school projects procured with CSP and CMR delivery system in the state of Texas. The study showed a 2.8% cost savings in CSP projects when compared with that of CMR projects. In a highway project studied by Shrestha et al (2011) showed an average increase of 6.3% cost growth in DBB project as opposed to a 7.8% in DB. That is a 1.5% cost saving in projects procured using DBB. Touran et al. (2011) also studied highway project that also resulted in a 0.5% cost growth for DBB project as compared to a 16.7% cost growth in CMR projects. Kulkarni et al. (2012) compared cost growth in 17 IPD-like projects (CMR) and 13 non-IPD like projects (CSP). The study showed a 26% cost saving for CSP projects as opposed to a 21% CMR projects. Neidert et al.

(2012) conducted a study on the use of CSP and CMR at the Texas A&M University System`s project. A total of 14 CSP and 19 CMR projects was analyzed and the result favorably showed a 3.7% cost saving for CSP projects as compared to 5.2% CMR projects. The Texas Legislative Council (2012) conducted a survey on the use of CSP, CMR, and DB on projects in the public sector. According to the report, CSP projects

53

resulted in a 23.1% cost saving compared to a cumulative 38.5% for CMR and DB projects. Like Kulkarni et al. (2012), In a study conducted by El Asmar et al. (2013) on IPD and non-IPD project over different states in the United States, the study p-value (0.471) showed a non-statistically significant cost growth in DBB/CSP as compared to the IPD project. Finally, Minchin et al. (2013) conducted a study on 51 highway projects that was completed between 2002 and 2010 in the state of Florida. According to the study and supporting FHWA (2006) and Shrestha et al (2011) observation, DBB showed almost 20% cost growth as opposed to 45% in DB. In order words, an approximate 25%

cost savings in projects procured with DBB project delivery method.

Overall, from table 5 above, like DB and CMR, very few studies did not showed DBB and CSP less favorably in terms of cost growth. Although, some studies did report statistical significance, few results reported were not statistically significant, and there were a few results that tend to be contradictory to typical findings on the topic. The studies in the above table for the most part shows that wherever DBB or CSP is benchmarked against DB and/or CMR or vice versa, they either or both resulted in a significant cost savings. Across the studies, it is identified that both DBB and CSP procurement type (lowest bidder and /or best value) has a significant influence on the overall project cost savings. This In order words can be described as a very low level of waste generated in terms of cost and a higher level of performance and benefit to the owner`s savings as compared to the other project delivery systems (DB and CMR). On

54

Key: TSG: Time/Schedule Growth; U/M: Univariate/ Multivariate; AV: Anova; N/A: Not Applicable; A/T-test: Anova/T-test.

55 4.1.2 Time/Schedule Performance

Like cost growth, the level of the delivery performance or waste is also quantified in terms of percentage by comparing the difference between the total built construction and total planned construction time to the total as planned construction time.

In a study conducted by Pocock (1996) on 25 public sector projects showed a non- statically significant 27.76.8% schedule growth in DBB projects as compared to 26.23%

in DB projects. Like the unfavorable DBB cost growth, Konchar and Sanvido (1998) study again resulted in 11.4% higher schedule growth for DBB as compared to DB. In a study conducted by Ibbs et al. (2003) on 67 building projects using data from the

in DB projects. Like the unfavorable DBB cost growth, Konchar and Sanvido (1998) study again resulted in 11.4% higher schedule growth for DBB as compared to DB. In a study conducted by Ibbs et al. (2003) on 67 building projects using data from the