Why Software Projects Escalate: The Importance of Project Management Constructs

(1)

Why Software Projects Escalate: The Importance of Project Management Constructs

Mark Keil, Member, IEEE, Arun Rai, Joan Ellen Cheney Mann, and G. Peter Zhang

Abstract—Previous research has documented that software projects are frequently prone to escalation. While the escalation literature acknowledges that project-related (as well as psycho- logical, social, and organizational) factors can promote escalation behavior, there has been no investigation regarding the role that project management factors may have in discriminating between projects that escalate and those that do not.

The objective of this study was to explore whether project man- agement constructs could be used to distinguish between projects that escalated and those that did not. Based on a survey admin- istered to IS audit and control professionals, data were gathered on projects that did not escalate as well as those that did escalate.

We then applied logistic regression to model the relationship be- tween various project management constructs and project escala- tion. The model was then evaluated for its ability to correctly clas- sify the projects.

The results of our research suggest that a logistic regression model based on project management constructs is capable of discriminating between projects that escalate and those that do not. Moreover, the model compares favorably to a previously published logistic regression model based on constructs derived from escalation theory. The implications of these findings are discussed.

Index Terms—Escalation, information technology, project man- agement, software.

I. INTRODUCTION

B

ASED on a survey of 376 CEOs, the consulting firm CSC Index, Cambridge, MA, reports that roughly 50% of all information technology (IT) projects fail to meet chief executives’ expectations [1]. Several books have appeared in recent years documenting such failures (see, for example, [2]–[4]) and Computerworld recently published a list of the top-ten corpo- rate IT failures [5].

In many cases, IT failures involve projects that go wildly over budget or drag on long past their originally scheduled com-

Manuscript received February 12, 2001; revised December 23, 2002. Review of this manuscript was arranged by Department Editor V. Sambamurthy. This work was supported in part by a grant from the Information Systems Audit and Control Foundation and in part by a research grant from the J. Mack Robinson College of Business, Georgia State University.

M. Keil is with the Department of Computer Information Systems, J. Mack Robinson College of Business, Georgia State University, Atlanta, GA 30303 USA (e-mail: [email protected]).

A. Rai is with the Electronic Commerce Institute, J. Mack Robinson Col- lege of Business, Georgia State University, Atlanta, GA 30303 USA (e-mail:

[email protected]).

J. E. C. Mann is with the MIS/DS, College of Business and Public Ad- ministration, Old Dominion University, Norfolk, VA 23508 USA (e-mail:

[email protected]).

G. P. Zhang is with the Department of Management, J. Mack Robinson Col- lege of Business, Georgia State University, Atlanta, GA 30303 USA (e-mail:

[email protected]).

Digital Object Identifier 10.1109/TEM.2003.817312

pletion date. Such projects have been labeled “runaway systems” in the trade press [6], [7]. Like a runaway train, these are projects that are hurtling out of control, difficult to stop, yet in need of redirection or termination. The behavior that underlies many runaway systems involves “escalation of commitment to a failing course of action” [8], a phenomenon that has been documented in the management literature and applied to software projects [9]–[12].

In order to prevent runaway projects, or at least minimize the costs associated with such projects when escalation does occur, it would be extremely useful to develop models capable of distinguishing between projects that are prone to escalation and those that are not. From a practical standpoint, this means investigating constructs that may be useful in discriminating between projects that escalate and those that do not.

Within the escalation literature, several theories have been proposed to explain the phenomenon. These include self-justification theory, prospect theory, agency theory, and approach- avoidance theory. In a recent paper by Keil et al. [13], constructs derived from these escalation theories were used as the basis for building logistic regression models that were then tested for their ability to correctly classify escalated and nonescalated software projects. While many of the constructs derived from various escalation theories did exhibit some predictive power in discriminating between projects that escalate and those that do not, there has been no comparable attempt to examine constructs associated with traditional project management and their ability to correctly classify such projects. Thus, while the escalation literature acknowledges that project-related (as well as psychological, social, and organizational) factors can promote escalation behavior, there has been no investigation regarding the role that project management constructs may have in discriminating between projects that escalate and those that do not.

In this paper, while we build on earlier work [13], [14], the focus and the research question driving the study, is different.

Keil et al. [13] examined only constructs associated with var- ious escalation theories; they did not explore the possibility that project management constructs might serve as escalation predictors. Drawing upon multiple theories of escalation, they followed a theory-by-theory approach to develop various logistic regression models and to test them for their ability to distinguish between escalated and nonescalated projects. Using a variable pool that included both escalation and project management con- structs, Zhang et al. [14] followed an atheoretical approach, con- structing separate neural network and logistic regression models to compare their classification ability. Their results suggest that neural network models can be powerful classifiers in the context of escalation. They also found that using a neural network

(2)

variable selection procedure, both project management and escalation theory variables entered into their model suggesting the importance of project management variables in explaining escalation. When they subsequently compared the relative predictive power of models based either on project management or escalation variables, the models based on project management variables proved to be better classifiers. A key limitation of their research is that it does not address the relative predictive power of specific project management constructs against each other and vis-à-vis specific escalation theory constructs. The lack of knowledge about the relative impact of project management and escalation constructs on escalation constrains the design of effective management interventions that prevent escalation from occurring.

In this paper, we explore the possibility of constructing a logistic classification model that is based solely on project management constructs. We also seek to understand the relative importance of project management variables in relation to each other and to escalation theory variables, in predicting escalation.

To summarize, the objective of this study was to explore whether project management constructs could be used to distinguish between projects that escalated and those that did not. The study was designed to answer the following two research questions.

1) Can project management constructs be used to create a logistic regression model capable of distinguishing between projects that escalate and those that do not?

2) Which project management constructs have the most predictive power in terms of distinguishing between projects that escalate and those that do not?

The remainder of this paper is organized into six sections.

First, we provide some background for our study by presenting a brief review of constructs derived from the project management literature. For each construct, we indicate how it may act to promote escalation. This is followed by a discussion of the research approach used to conduct the study. Next, we discuss construct reliability and validity, followed by a brief overview of the logistic regression modeling approach we used for our analysis. The actual model building process is then described, along with the results obtained. Finally, we discuss the limitations of our study and its implications for both research and practice.

II. BACKGROUND ANDLITERATUREREVIEW

Escalation occurs when troubled projects are continued instead of being abandoned or redirected. Several researchers have applied the concept of escalation to explain the prevalence of runaway systems projects. While escalation is most frequently thought of as a “bad” thing, it should be noted that there may be cases in which escalation is warranted (see, for example, [15]).

In this paper, however, we focus on cases in which escalation, as judged through the eyes of a trained professional—an information systems (IS) auditor—is unwarranted. Constructs derived from the project management literature were explored as possible predictors of IT project escalation.

A. Constructs Derived From the Project Management Literature

The project management literature suggests that poor project management practices can promote runaway systems projects.

Since many runaway systems projects exhibit the characteris- tics associated with project escalation, it stands to reason that project management constructs may represent a viable approach for discriminating between projects that escalate and those that do not.

The project management arena is a broad one. To highlight the area of focus for this study, we draw upon the work of Schmidt et al. [16] who propose an IT project risk classification framework. Specifically, they distinguish between “inside” risks and “outside” risks. Inside risks are those that can be monitored and controlled by the project manager. Outside risks are those that the project manager has limited ability to control or influ- ence. Here, we focus on what Schmidt et al. [16] term as “inside risks” (i.e., those over which the project manager has a relatively high degree of control). The inside risks can be broadly grouped into two areas: project scope and requirements and project execution [17]. Within these areas, the information systems literature (see, for example, [18]–[20]), points to a handful of key project management constructs that have been consistently associated with the ultimate failure or success of software projects.

These include: project planning, project specification, project estimation, project monitoring, and project control. While we make no claim to have exhaustively covered the full range of constructs described in the project management literature, these particular constructs were selected and retained for further study for two reasons. First, they are central to the project management body of knowledge, and second, a logical argument can be made that they may be connected to the phenomenon of project escalation.

B. Project Planning

Planning processes are central to project management and are defined as such by the Project Management Institute’s (PMI) guide to the project management body of knowledge (PMBOK).

Indeed, planning is so central that it cuts across all nine core bodies of project management knowledge identified by PMI.

The PMBOK defines planning as “defining and refining objectives” [21]. IS professionals agree that projects often fail because too little attention is paid to planning issues before the project is initiated. The literature suggests that project goals and objectives are often ill-defined [21], [22].

A well-defined project plan should include a schedule of milestones to be met and deliverables to be completed. In some IT projects, a plan is never created at all. In other cases, the planning process is too loose and informal to create an effective plan [19], [23]–[25]. In addition to creating a viable plan, the planning process should also develop success criteria for deliverables, assign responsibilities to project participants and create contingency plans based on potential risk factors [20], [24], [26], [27]. If there is no project plan or no success criteria for deliverables, then there is no way to know whether the project is performing well. When responsibilities are not clearly designated, then there is no accountability. This means there

(3)

is no way to be sure tasks are being handled well by project participants, and when trouble starts, no way to determine who is responsible. Lack of attention to planning issues can promote project escalation and ultimately lead to project failure. Poor project planning makes setbacks more likely to occur and less likely to be noticed and properly dealt with when they do occur.

Thus, poor planning may promote escalation behavior.

C. Project Specification

Project specification is an important aspect of project scope management which is defined by PMI to be one of the core project management knowledge areas [21]. The PMBOK defines scope management as ensuring that “the project includes all the work required, and only the work required, to complete the project successfully” [21, p. 189]. In the context of information technology projects, scope management involves defining users’ needs so that the requirements can be specified accurately

1. In the extreme case, failure to define accurate specifications can lead to the creation of a system that does not fit the users’

needs, thus resulting in project failure [26]. The most often cited reason for inadequate and/or inaccurate specifications is lack of involvement from users [18], [20], [23], [28]. Still, even when users are involved, poor specifications can result if information system personnel fail to adequately analyze user needs and the organizational environment in which the system will be used [18], [20], [26], [29]. In either case, failure results from inadequate specifications because the system created will not adequately address users’ needs. Poor specification can result in constant changes to requirements, with each change leading the decision-maker(s) into a false sense of security about having

“gotten it right this time.” Thus, poor specification may promote a kind of creeping escalation of commitment.

D. Project Estimation

Project estimation is a critical part of project time management which is defined by PMI to be one of the core project management knowledge areas [21]. The PMBOK defines time management as “ensuring timely completion of the project” [21, p. 190]. A central feature of time management is developing the activity duration estimates needed to create the project schedule.

The work breakdown structure (WBS) serves as a means of enu- merating and organizing the activities and work packages that must be performed in order to complete the project. Without a solid WBS and good duration estimates for the activities involved, it is impossible to develop a credible project schedule.

In spite of this project management truism, IT projects are rou- tinely executed without adequate attention toward developing a solid WBS and accompanying duration estimates. The result is that estimates of the time and resources required to successfully complete the project are grossly underestimated from the outset.

Without an understanding of the size, complexity, and scope of a project, schedules are meaningless.

Before a project begins, it is important to estimate the time and resources needed to complete the project. Underestimation,

1In IT projects, another aspect of scope management involves the prevention of what is often termed “scope creep.” This aspect of scope management was not included in our operationalization of the project specification construct.

however, is one of the most often cited causes of failure men- tioned by information system professionals [19], [22], [29], [30]. Chronically underestimating the scope and complexity of information systems projects may promote escalation by preventing decision-makers from distinguishing between

“normal” deviations from plan and more serious deviations from plan.

E. Project Monitoring and Control

Monitoring is defined as “the capture, analysis, and reporting of project performance, usually as compared to plan” [21, p.

203]. It involves regularly evaluating a project and being alert to potential problems that might cause it to fail. Monitoring for risks involves the examination of residual risks and the identi- fication of new risks that may emerge during project execution.

Projects often fail because managers or executives fail to adequately monitor the project and, therefore, problems are not noticed or addressed until it is too late to bring the project back on track [19], [20]. Poor monitoring may encourage escalation by allowing a troubled project to continue to absorb resources when it might otherwise be terminated or significantly redirected.

Controlling processes are central to project management. In- deed, control is so central that it cuts across seven of the nine core bodies of project management knowledge identified by PMI. The relevant portion of the PMBOK definition that deals specifically with control (as opposed to monitoring) suggests that controlling processes are adjustments to the project plan that are made in response to significant variances that are de- tected through monitoring of the project [21, p. 36]. Control activities are undertaken after monitoring has uncovered problems in the information systems development project [19], [20]. The goal of control activities is to get the project back on track. More broadly, controlling also includes taking preventive action in an- ticipation of possible problems [21, p. 36].

While subtle distinctions can be drawn between monitoring and control, the PMBOK and other sources tend to combine the notions of monitoring and control as subdimensions of the same construct. Hence, we elected to treat it is a single construct here.

Without effective monitoring and control, a troubled project is likely to either go undetected or uncorrected. Thus, poor monitoring and control may contribute to project escalation.

Table I lists the constructs derived from the project management literature and describes how they may serve to promote project escalation (see [31] for more detail on the arguments summarized in the table).

III. RESEARCHAPPROACH

In order to address our research questions, a cross-sectional survey of information systems audit and control professionals was used to collect information on both escalated and nonesca- lated projects. Using data collected as part of the study reported in Keil et al. [13], we examine constructs derived from the project management literature to determine whether a model comprised of such constructs can predict escalation.

The subject pool for the study consisted of 2231 IS audit and control professionals in the U.S. who were registered as mem- bers of the Information Systems Audit and Control Association

(4)

TABLE I

KEYCONSTRUCTSDERIVEDFROM THEPROJECTMANAGEMENTLITERATURE ANDTHEIRRELATIONSHIP TOESCALATION

(ISACA). In order to provide a control group for comparison purposes, two different versions of the survey instrument were developed: one designed to gather information on cases of software project escalation and the other designed to gather infor- mation on projects that did not escalate.

Participants who received the “escalation” version of the survey were asked to identify a troubled project that continued to receive resources even though s/he thought that the project should have been discontinued or redirected. Participants who received the “nonescalation” version of the survey were asked to select a project that had progressed smoothly enough that the respondent never thought it should be discontinued or redirected. For the full-scale administration of the survey, 70%

of the participants received the escalation version of the survey and 30% received the nonescalation version of the survey²

A. Response Rate and Respondent Demographics

In total, 579 surveys were returned, yielding a 26% overall response rate. Seventy-two percent of our respondents indicated that they were either an IS auditor or IS audit manager. The remaining respondents identified themselves either as external auditors (12%) or specified a different audit-related title (16%).

Survey respondents had an average of 8.7 years of experience as an information systems audit and control professional.

The respondents represented organizations varying in size from 10 000 or more employees (36%) to fewer than 100 employees (1%). Most respondents represented medium to large organizations with 1000 or more employees (83%). Forty per-

2For more detail on the design, sampling procedure, pretesting, and piloting of the survey, see Keil, Mann, and Rai [13] and Mann [31].

cent of the respondents worked for organizations in the financial services industry, though a wide variety of other sectors were also represented, including government (15%), manufacturing (13%), utilities (6%), trade (6%), health care (4%), transporta- tion (2%), education (2%), and other (12%).

With respect to the projects they reported on, nearly half of the subjects indicated that they had either served as the project manager (5%), as a member of the development team (11%), or had been responsible for evaluating the project on a regular basis (32%). Approximately half of the respondents indicated that they became associated at a very early stage with the projects they reported on, either during planning (24%) or requirements analysis (24%).

Nonresponse bias testing was performed as described in Keil et al. [13] and no evidence of nonresponse bias was found. Of the surveys returned, approximately 58% were completed with specific information about a particular project that had either escalated or not escalated. These contained information regarding both the escalation and project management factors that were present or absent during the course of the project and were retained for subsequent analysis.

IV. CONSTRUCTRELIABILITY ANDVALIDITY

To increase the reliability of the survey instrument, multiple measurement items were initially developed for each construct.

The operationalization of constructs derived from the project management literature was based on a review of the literature and its application to the domain of software project management [31]. Table II shows the actual measurement items that were used to operationalize the constructs.

(5)

TABLE II

CONSTRUCTOPERATIONALIZATION

A. Reliability Assessment for Project Management Constructs All project management constructs were measured using multiple item measures, which were assessed for reliability.

Four of the six constructs were measured using scales with two dichotomous measurement items, whereas the other two constructs were measured using scales with four and five dichotomous measurement items. As all of our measurement items were dichotomous, standard measures of reliability, such as Cronbach’s alpha, could not be used. Instead, we assessed the degree to which items associated with a given scale exhibited complete conformity, partial conformity, or no conformity.

Complete conformity occurs when coded responses are identical for all items associated with a scale. Partial conformity occurs in scales with three or more items when the coded response on one item is different from the coded response on the other items. Nonconformity occurs in two-item scales when the coded

response on one item is different from the coded response on the other item. By definition, two-item scales cannot exhibit partial conformity. Table III summarizes the level of conformity associated with each of the measurement scales.

B. Convergent and Discriminant Validity for Project Management Constructs

The item-to-construct correlations were first computed to examine convergent and discriminant validity (Table III).

Table IV shows good discriminant validity with low cross- correlations between indicators and constructs other than the ones they were designed to measure. Although the items for planning and monitoring and control showed moderate cross- correlations, they correlated more strongly with their intended construct. Where moderate cross-correlations did exist, they did

(6)

TABLE III

ITEMCONFORMITY FORPROJECTMANAGEMENTCONSTRUCTS

TABLE IV

ITEM-TO-CONSTRUCTCORRELATIONS FORPROJECTMANAGEMENTCONSTRUCTS

The actual scale items used are shown in Table II. Shaded cells in this table represent insignificant correlations (p > 0:05).

TABLE V

INTER-CONSTRUCTBIVARIATECORRELATIONS FOR PROJECTMANAGEMENTCONSTRUCTS

not approach the threshold of 0.707 which would indicate 50%

shared variance. To further assess the discriminant validity of our project management constructs, we examined the intercon- struct correlation matrix (Table V).

While planning exhibited a moderately high correlation with monitoring and control, we elected to retain both constructs as they are conceptually distinct and the level of correlation is on the margin of what is considered acceptable from a discriminant validity standpoint.

After investigating the reliability and validity of our measures, aggregate measures were computed by summing responses to scale items and then dividing by the number of items. Since each item was measured on a dichtomous scale (1 if the item was present in the project and 0 if it was not), the values of each construct represented a continuum that varied from 0 (not present) to 1 (present).

V. MODELBUILDING ANDRESULTS

Logistic regression analysis was chosen over discriminant analysis, because we are dealing with a dichotomous dependent variable, and nonparametric type independent variables with nonnormal distributions. The dependent variable was whether the project was escalated (1) or nonescalated (0). Each independent variable was whether a condition was present in the project (1) or not present (0). Logistic regression uses a nonparametric model and, therefore, does not require variables to be normally distributed.

A. Controlling for Exogenous Variables

Demographic variables, such as organization and IS department size, auditor experience and project size, can potentially impact escalation. The sample was segmented using each of these variables but no significant differences were found across escalated and nonescalated projects, except for project size.

Project size, therefore, was used as an exogenous variable in the analysis. As shown in TableII, respondents indicated the relative size of the project on a Likert scale, which ranged from 0 (smaller in size compared to other IS projects undertaken) to 6 (larger in size compared to other IS projects undertaken). Size was included to examine if larger projects were more vulner- able to escalation. A normalized measure of project size was computed by dividing indicated responses by six.

(7)

TABLE VI

ANALYSIS OFPROJECTESCALATIONBEHAVIORRESULTS FORMODELFIT

B. Logistic Regression Results

Our first step was to take the project management constructs and create an estimation model with those constructs and project size using 75% of the original sample, which yielded 242 cases that were randomly selected for analysis. The significance of the model was then evaluated and the importance of each construct within the model was assessed. Neither planning nor size were found to be significant variables in the logistic regression model

( and , respectively). These constructs

were, therefore, dropped from further analysis. The estimation model was rerun using the remaining variables, again using 75%

of the original sample (using new randomly selected cases to generate the model). The monitoring and control, estimation, and specification constructs were all found to be significant in this model. Finally, the other 25% of the original sample was used to test the models predictive validity by determining how useful the model was in classifying new projects.

In logistic regression, several measures of model significance can be used. Table VI shows all the fit measures for our model based on different project management constructs. For comparison purposes, Table VI also includes logistic regression results reported by Keil et al. [13] using constructs derived from various escalation theories. TableVII provides an explanation of relevant statistics used for interpreting logistic regression results.

According to the Hosmer–Lemeshow Goodness of Fit Test, the project management model was significantly better at determining the probability of project escalation than random chance.

It also explained more of the variance than any of the previously reported models based on constructs derived from escalation theory.

To continue the assessment of the model, TableVII presents several measures describing the importance of constructs within each model. The significance of a construct was assessed by examining the -value of the Wald statistic (which measures significance of the construct after accounting for its error) and the 95% confidence interval odds ratio (if the interval does not contain the number one, then the item is significant). As shown in TableVII, the project management constructs (specification, estimation, and monitoring and control) were all significant with Wald -values less than 0.001 and 95% confidence interval odds ratios that did not include one. For comparison purposes, TableVII includes logistic regression results reported by Keil et al.[13] using constructs derived from various escalation theories.

The degree to which a construct was useful for distinguishing project escalation was determined by examining the standard-

ized betas (SE Beta) and the odds ratios. The odds ratio is the increase in likelihood in project escalation corresponding to an increase in an independent variable. In the model based on project management constructs, the monitoring and control variable was the most important with a standardized beta of 2.97 and it exhibited the highest odds ratio suggesting that there is nearly a 20-fold increase in the likelihood of project escalation under conditions of poor monitoring and control.

In comparing the results of the project management model against those obtained with various escalation-related constructs, we first note that project size was an important variable in the models derived from escalation theory constructs, but that it does not even enter into the model based on project management constructs. We suspect that this may be because one or more of the project management constructs is also serving as a proxy for project size. In terms of odds ratios, the variables that best distinguish between escalated and nonescalated projects are monitoring and control and the completion effect which both exhibit odds ratios of about 20. As TableVII indicates, the odds ratios of other variables that were found to be significant were much lower, ranging in value from 3.39–8.72. Interest- ingly, the other two variables that were significant in the project management model—specification and estimation—exhibited odds ratios that were greater than the odds ratios of most of the escalation-related constructs including: sunk cost effect, goal congruency, and information asymmetry.

C. Predictive Validity—How Well Does the Model Classify New Projects?

To assess the predictive validity of the model, we examined its classification performance on both the estimation sample and a separate holdout sample. As a base of comparison, we computed the Morrison’s [32] proportional chance criterion given by the model

where alpha is the proportion of projects in group 1 and is the proportion of projects in group 2. In our study, group 1 was escalated projects (approximately 73%) and group 2 was nonescalated projects (27%). For our sample, Morrison’s chance criterion (i.e., the overall percentage correct expected by chance) is 0.59. Accordingly, we used a cutoff estimated probability of 0.59 to determine membership in a given group according to chance.

(8)

TABLE VII

EXPLANATION OFRELEVANTSTATISTICS FORINTERPRETINGLOGISTICREGRESSIONRESULTS

Table VIII shows the classification results that were obtained for both the estimation sample and the holdout sample.

Chi-square tests indicated that the logistic regression model based on project management constructs performed better than chance in terms of its classification ability for both the estimation and holdout samples (as did the model(s) based on constructs derived from escalation theories).

As shown in Table IX, the model based on project management constructs had better predictive validity for escalated projects than did any of the models based on escalation theory constructs. The project management model also had greater predictive validity in the holdout sample for nonescalated projects than any of the models based on escalation theory. In terms of overall classification ability, the project management model outperformed all of the escalation theory models in both the estimation and holdout samples. The strongest of the escalation theory models was the one based on approach avoidance and the completion effect which exhibited an overall classification rate of 75% on the holdout sample [13].

In comparison, the project management model exhibited an overall classification rate of 91% on the holdout sample. As this performance level is consistent across the estimation and holdout samples, we have strong evidence of the validity of project management constructs in predicting project escalation.

VI. LIMITATIONS OF THESTUDY

Before discussing the implications of our work, it is appro- priate to mention its limitations. This study relied on IS audit

and control professionals as subjects and each case involved self-reported information from a single respondent concerning a project that had been completed some time earlier. This raises the prospect of a possible bias or a significant amount of error in our data set and, thus, our conclusions must be interpreted with some caution.

While the choice of auditors as subjects certainly creates the possibility of bias, there were several advantages of relying on IS auditors that outweighed these concerns. These include the following:

1) IS auditors do not have directly vested interests in project outcomes because their careers are unlikely to be made or broken by a project’s success or failure;

2) IS auditors can be expected to report more objectively than managers and other project participants;

3) they have access to objective data on project performance;

4) they have experience with multiple projects and formal standards for assessing projects.

Another limitation of our study is that we have focused on a relatively narrow range of project management constructs. It is quite possible that other project management constructs not explored here might also be useful in predicting project escalation. In recent years, for example, there has been considerable emphasis on developing measurable and repeatable processes for software development. The Software Engineering Institute’s Capability Maturity Model (CMM) is one in which process-related factors are of central importance. Future research is needed to determine if process-related factors, or other project management variables, can be used to predict escalation. Future re-

(9)

TABLE VIII

ANALYSIS OFPROJECTESCALATIONBEHAVIORSIGNIFICANCERESULTS FOREACHCONSTRUCT

TABLE IX

RESULTS OFLOGISTICREGRESSIONMODELS—CLASSIFICATIONANALYSIS

search is also needed to determine if there is value in developing a composite model that combines project management constructs with escalation theory constructs.

VII. IMPLICATIONS ANDCONCLUSION

Constructs derived from the project management literature were shown to be useful in classifying projects. In comparing our results against those reported by Keil et al. [13], the model we developed here based on project management constructs appears to be a better classifier than the models they developed based on constructs derived from various escalation theories.

This result is significant because project management variables are easy to measure and relate to concepts that managers can easily understand. Moreover, there is a wealth of project management knowledge that already exists and can be brought to bear concerning the key variables that are most predictive of project escalation.

The most important project management variables identified in this study are: specification, estimation, and monitoring and control. These are the variables that seem best at discriminating between escalated and nonescalated projects. The results of our study have important implications for both research and practice.

A. Implications for Research

First, the study suggests that project management constructs can be used to build a model capable of distinguishing projects that escalate from those that do not. Interestingly, it appears that such a model is actually better than previous models that have been developed based on constructs derived from various escalation theories. This represents a significant finding in that escalation research has largely ignored the project management literature in developing constructs and theories that might explain and predict escalation behavior. This study, by comparing the classification capability of models based on escalation

(10)

theory constructs and project management constructs, clearly shows that both sets of constructs can yield useful classification models, but that those based on project management constructs tend to be better classifiers.

Our findings suggest that escalation is a complex phenomenon and that more sophisticated models are needed to more fully explain escalation behavior. From a research perspective, this suggests that more attention should be placed on developing richer theories of escalation that incorporate project management constructs.

The results we obtained indicate that it is possible to predict escalation with a fairly high degree of accuracy using a model based solely on project management constructs. It should be noted, however, that the model was somewhat better at predicting escalated projects than nonescalated projects, which raises the inherent tradeoff between Type I and Type II errors associated with any classification model. Specifically, our results suggest that the use of the model will likely result in more Type II errors than Type I errors; in other words, the detection of nonescalated projects may be compromised in favor of the detection of escalated projects. Given the massive expenditures and negative organizational outcomes associated with project escalation, such a tradeoff may be more desirable than an increase in the Type I error rate, i.e., a superior ability to detect nonescalated projects with a decrease in ability to detect escalated projects. Users of this model can be fairly confident if the model identifies an escalated project, but should be somewhat more cautious with respect to a nonescalated prediction.

While the project management model developed here was better able to classify cases of escalation, we do not mean to imply that the escalation constructs are unimportant. It is important to recognize that some of the escalation theory constructs (e.g., self justification) are difficult to operationalize and measure directly [13]. Thus, it may be that these constructs are equally important, but that is because of the difficulty of measuring them they are less predictive of escalation than one might think. It is our speculation that the escalation theory constructs may be important predictors for more serious and advanced cases of escalation; they may, however, be less visible and less important predictors for more mild cases of escalation or for projects that are at a relatively early stage of development. This would appear to represent an interesting avenue for further research.

In a related vein, future research should investigate if there are different predictors of escalation in different contextual set- tings. Interesting contextual differences can occur due to the nature of the stakeholders or the nature of the project itself. For example, projects involving multiple stakeholders, within and across companies, may escalate under a different set of conditions in comparison to projects whose stakeholders are relatively homogeneous in their goals and objectives. Similarly, projects focused on platform capabilities may escalate under a different set of conditions in comparison to specific business application projects. Such investigations focused on examining the moder- ating effect of type of escalation (mild or severe), type of stakeholders (homogeneous of diverse with respect to goals and objectives), and type of project (platform initiative or specific busi-

ness application) will add to our understanding of this complex phenomenon.

B. Implications for Practice

From the standpoint of practice, the results that were obtained in this study suggest that practitioners need not neces- sarily measure the psychological, social, and organizational factors associated with escalation to determine if a project is headed for trouble. Indeed, our research has shown that from a prag- matic standpoint, managers may be able to predict escalation more accurately solely on the basis of constructs derived from project management. It would also appear that accurate specification and estimation, along with good project monitoring and control, would go a long way toward reducing the incidence of project escalation. This represents an important set of findings for two reasons. First, it suggests that measuring traditional project management variables can prove a useful diag- nostic aid to the practitioner in identifying projects that may be prone to escalation. Second, it suggests that managers may be able to reduce the frequency of project escalation simply by implementing good project management techniques in a few key areas.

Specifically, our results suggest that managers who are willing to invest in project management tools, techniques, and training, particularly in the areas of specification, estimation, monitoring and control could reap substantial benefits in terms of reducing the incidence of runaway IT projects. The project management and software engineering literatures are good sources for techniques and approaches that can improve the specification (e.g., JAD, RAD, and extreme programing), estimation (e.g., COCOMO, function points), monitoring and control (e.g., earned value analysis) of software projects.

Practitioners interested in detecting and preventing software project escalation should consider using the model developed here based on project management constructs. This model classified nearly 95% of the escalated projects and approximately 83% of the nonescalated projects correctly (in the holdout sample). By using the model, practitioners should be able to more readily identify projects that would be prone to escalation, so that corrective action can be taken. However, the model should be used with some caution, as its classification rate for nonescalated projects was not nearly as good in the estimation sample. In other words, use of such a model may result in a significant number of nonescalation projects being classified as cases of escalation. For organizations and managers that are risk averse, this may not be a drawback as this type of error in classification may be viewed as more desirable than incorrectly classifying a nonescalation project as a case of escalation.

In summary, practicing managers should consider using and refining this model as an aid in reviewing their software project portfolios for possible cases of software project escalation.

REFERENCES

[1] B. Wysocki, “Some firms, let down by costly computers opt to ‘de-en- gineer’,” Wall Str. J., p. A1, A6, 1998.

[2] S. Flowers, Software Failure: Management Failure. New York: Wiley, 1996.

(11)

[3] R. L. Glass, Software Runaways. Englewood Cliffs, NJ: Prentice-Hall, 1998.

[4] , Computing Calamities. Englewood Cliffs, NJ: Prentice-Hall, 1999.

[5] K. Nash, “Top 10 disasters,” Computerworld, vol. 36, pp. 1, 32–33, Oct.

30, 2000.

[6] M. Mehler, “Reining in runaways,” Inform. Week, pp. 20–24, Dec. 16, 1991.

[7] J. A. Willbern, “Runaway computer systems,” Best’s Rev. (Life/Health), vol. 89, pp. 90–92, 1989.

[8] J. Brockner, “The escalation of commitment to a failing course of action:

Toward theoretical progress,” Acad. Manage. Rev., vol. 17, pp. 39–61, 1992.

[9] M. Keil, “Pulling the plug: Software project management and the problem of project escalation,” MIS Quart., vol. 19, pp. 421–447, 1995.

[10] M. Keil, R. Mixon, T. Saarinen, and V. Tuunainen, “Understanding runaway information technology projects: Results from an international re- search program based on escalation theory,” J. Manage. Inform. Syst., vol. 11, pp. 67–87, 1995.

[11] M. Newman and R. Sabherwal, “Determinants of commitment to infor- mation systems development: A longitudinal investigation,” MIS Quart., vol. 20, pp. 23–54, 1996.

[12] H. Drummond, Escalation in Decision-Making: The Tragedy of Taurus. Oxford, , U.K.: Oxford Univ. Press, 1996.

[13] M. Keil, J. Mann, and A. Rai, “Why software projects escalate: An em- pirical analysis and test of four theoretical models,” MIS Quart., vol. 24, pp. 631–664, 2000.

[14] G. P. Zhang, M. Keil, A. Rai, and J. Mann, “Predicting information technology project escalation: A neural network approach,” Eur. J. Oper.

Res., 2003, to be published.

[15] M. Keil and J. Flatto, “Information systems project escalation: A reinter- pretation based on options theory,” Account., Manage. Inform. Technol., vol. 9, pp. 115–139, 1997.

[16] R. Schmidt, K. Lyytinen, M. Keil, and P. Cule, “Identifying software project risks: An international delphi study,” J. Manage. Inform. Syst., vol. 14, pp. 5–36, 2001.

[17] M. Keil, P. E. Cule, K. Lyytinen, and R. C. Schmidt, “A framework for identifying software project risks,” Commun. ACM, vol. 41, pp. 76–83, 1998.

[18] S. Alter and M. Ginzberg, “Managing uncertainty in MIS implementa- tion,” Sloan Manage. Rev., vol. 20, pp. 23–31, 1978.

[19] M. M. Jones and E. McLean, “Management problems in large-scale soft- ware development projects,” Ind. Manage. Rev., vol. 11, pp. 1–15, 1970.

[20] D. P. Slevin and J. K. Pinto, “The project implementation profile: New tool for project managers,” Proj. Manage. J., vol. 17, pp. 57–70, 1986.

[21] A Guide to the Project Management Body of Knowledge. Newtown Square, PA: Project Management Institute (PMI), 2000.

[22] D. D. Phan and J. Nunamaker, “The search for perfect project manage- ment,” Computerworld, pp. 95–100, Sept. 26, 1988.

[23] J. W. Schmitt and K. A. Kozar, “Management’s role in information system development failures: A case study,” MIS Quart., vol. 2, pp.

7–16, 1978.

[24] R. H. Thayer, A. Pyster, and R. C. Wood, “Validating solutions to major problems in software engineering project management,” Computer, vol.

15, pp. 65–77, 1982.

[25] R. F. Powers and G. W. Dickson, “MIS project management: Myths, opinions, and reality,” Calif. Manage. Rev., vol. 15, pp. 147–156, 1973.

[26] K. Lyytinen, “Expectation failure concept and systems analysts’ view of information system failures: Results of an exploratory study,” Inform.

Manage., vol. 14, pp. 45–56, 1988.

[27] H. Barki, S. Rivard, and J. Talbot, “Toward an assessment of software development risk,” J. Manage. Inform. Syst., vol. 10, pp. 203–225, 1993.

[28] G. Gunawardane, “Implementing a management information system in an extremely dynamic (and somewhat hostile) environment—A case study,” Interfaces, vol. 15, pp. 93–99, 1985.

[29] H. L. Morgan and J. V. Soden, “Understanding MIS failures,” Data Base, vol. 5, pp. 157–167, 1973.

[30] T. Abdel-Hamid and S. E. Madnick, Software Project Dynamics: An Integrated Approach. Englewood Cliffs, NJ: Prentice-Hall, 1991.

[31] J. Mann, The Role of Project Escalation in Explaining Runaway Infor- mation Systems Development Projects: A Field Study. Atlanta, GA:

Dept. Comput. Inform. Syst., Georgia State Univ., 1996.

[32] D. G. Morrison, “On the interpretation of discriminant analysis,” J.

Market. Res., vol. 6, pp. 156–163, 1969.

Mark Keil (M’02) received the B.S. degree from Princeton University, Princeton, NJ, the M.S.

degree from the Sloan School of Management, Massachusetts Institute of Technology, Cambridge, and the Ph.D. degree in management information systems from the Harvard Business School, Harvard University, Cambridge, MA.

He is currently a Professor of Computer Infor- mation Systems (CIS) in the J. Mack Robinson College of Business, Georgia State University, Atlanta. His research interests include software project management with particular emphasis on identifying and preventing software project escalation. His publications have appeared in such journals as IEEE TRANSACTIONS ONENGINEERINGMANAGEMENT, MIS Quarterly, Sloan Management Review, Communications of the ACM, Journal of Management Information Systems, and Decision Support Systems. He has also served as an Associate Editor for MIS Quarterly, and as Co-Editor of The DATA BASE for Advances in Information Systems.

Dr. Keil currently serves on the editorial board of IEEE TRANSACTIONS ON ENGINEERINGMANAGEMENT.

Arun Rai is a Harkins Chaired Professor in the e-Commerce Institute and Computer Information Systems Department, the J. Mack Robinson College of Business, Georgia State University, Atlanta. His research interests include the diffusion, infusion, and impacts of information technology, digitally enabled supply chain management, and management of collaboration processes such as distribution, logistics, innovation, product development, and systems development. He has published several articles on these and related subjects in journals including Accounting, Management and Information Technologies, Communications of the ACM, Decision Sciences, Decision Support Systems, European Journal of Information Systems, Information Systems Journal, Information Systems Research, Journal of Management Information Systems, MISQ, and Omega.

He previously served on the editorial boards of Information Systems Research and MIS Quarterly, among others

Dr. Rai currently serves on the editorial board of the IEEE TRANSACTIONS ON ENGINEERINGMANAGEMENT.

Joan Ellen Cheney Mann received the Ph.D. degree in management informa- tion systems from Georgia State University, Atlanta.

She is currently an Assistant Professor at Old Dominion University, Norfolk, VA. Her research has been published in several journals, including MIS Quar- terly, European Journal of Operations Research, and Journal of IT Education, as well as, many national conferences including the Association for Informa- tion Systems, Academy of Management, and Decision Science Institute. Her current research interests include Runaway IS projects, Global IT issues, and other management of information topics.

G. Peter Zhang received the Ph.D. degree in operations research/operations management from Kent State University, Kent, OH, in 1998.

He is currently an Associate Professor of Man- agement at Georgia State University, Atlanta. His research has appeared in IEEE TRANSACTIONS ONNEURALNETWORKS, IEEE TRANSACTIONS ON SYSTEMS, MAN,ANDCYBERNETICS, Computers and Industrial Engineering, Computers and Operations Research, Decision Sciences, European Journal of Operational Research, IIE Transactions, Interna- tional Journal of Forecasting, International Journal of Production Economics, Journal of the Operational Research Society, Neurocomputing, among others.

His current research interests include neural networks, forecasting, time series analysis, and supply chain management. He is on the editorial review board of Production and Operations Management.