Managing and Valuing a Corporate IT
Portfolio Using Dynamic Modeling of
Software Development and Maintenance
Daniel Goldsmith, Research Scientist, MIT Sloan School of Management
Dr. Michael Siegel, Principal Research Scientist Sloan School of Management
The goal of this research is to enable performance improvements in IT portfolio management. Through investigation of software practices at a Fortune 500 company, we were able to demonstrate how simulation analysis can improve the valuation of software applications and overcome difficulties in reducing software maintenance costs. Our analysis used the System Dynamics Modeling (SDM) methodology to develop and simulate quantitative models that linked software dynamics, key actors, and management systems to estimate the costs and benefits of various management approaches. We also utilized complementary research techniques ‐ including onsite interviews with multiple constituencies ‐ to develop a holistic view of the software maintenance/development cycle.
The results of this research are being applied directly in developing new approaches to software maintenance and development at the study company, and we believe this research is applicable to other companies in various industries. At its core the research uses new simulation modeling techniques that can help organizations best manage their technology portfolio and process improvement efforts.
Dynamics of the Software Lifecycle
Why are software lifecycle costs often larger than expected and hard to reduce? How can we overcome organization difficulties in managing technology development and maintenance? The
answers to these questions must take into account the dynamics of the IT portfolio. The IT portfolio is the system of connections that link critical processes, such as application development and maintenance, in underappreciated ways that ultimately determine portfolio performance. To explore these dynamics and demonstrate their crucial role in valuing and managing the IT portfolio, we researched the practices of a Fortune 500 company in managing its software maintenance process. In particular, while a significant percent of software costs occurred during the maintenance phase, and despite best efforts to address them, maintenance costs were proving difficult to predict, manage, and reduce.
We began by mapping a simplified view of the organizational resources and areas of focus for the application development and maintenance phases, shown in Figure 1. As introductory definitions1, by “development” we generally mean the process of architecting, designing, implementing, and deploying software. By “maintenance” we mean the process of correcting, adapting, and perfecting that software.
Timeline: Application Lifecycle Costs
Organizational Resources Development Maintenance Development 20-40 Percent 60-80 Percent Business Opportunity Time to Market Business Requirements Maintenance Business Opportunity Time to Market Business Requirements Value Of Business -Resource Cost -Lost Opportunity Cost = Net Business Value Value Of Business -Resource Cost -Lost Opportunity Cost = Net Business Value
Figure 1. Development and Maintenance
There exists a large literature on software management practices themselves. (See IEEE. 1993 for definitional approach)This work focuses on underreported linkages and dynamics surrounding these practices.
The figure above depicts the organizational resource “pie” and the factors that drive decisions concerning development (on average 20‐40 percent of cost at the beginning of the lifecycle) and maintenance (on average 60‐80 percent of cost at the end of the lifecycle. (Takang and Grubb 1996) For development, key concerns include: the business opportunity from new application development, the desired time to market, and the overall business requirements supported by the application. For maintenance, key concerns include: the value of business from the application to be maintained, the resource cost invested in maintenance, and the potential lost opportunity from investing resources in maintenance which could be alternatively be used to development new applications.
All of these elements help derive the net business value of an individual application and a firm’s overall IT portfolio. However, organizations often fail to include all these critical factors in the planning and budgeting for their IT portfolio. Managers may choose courses of action that appear to increase business value in the short run, but which only decrease value in the long run. Further, these decisions may lead to compounding or cascading effects, which cause a series of negative unintended consequences. We next describe several of these situations which occurred at the company of study.
The Situation at the Company
Company executives were concerned because they were faced with rising maintenance costs in an environment in which costs were expected to shrink. While they had previously tried several approaches to reduce costs, the approaches had not proven effective in the long‐term. They had predominately focused on maintenance specifically (such as benchmarking, better data collection, and new reporting and management systems), but had not focused on the linkages between development and maintenance, nor in the business units and organization as a whole. An initial insight of our research was the ability to show the necessity of thinking about development and maintenance as a series of integrated process, with effects going both ways (particularly overlooked was the possibility of
growing maintenance budgets reducing development budgets).
Through a series of onsite interviews with multiple constituencies, we uncovered the following explanations as to why costs were rising:
Rushed Development: A business unit would push for quick development of an application, focusing on the potential business value from a new application and a rapid time to market, but would underestimate the long‐term increase in maintenance that would result. By way of demonstration, it is possible that a rushed development project could have a maintenance “tail” of ten times that originally estimated, overwhelming any of the projected gains from the new application.
Imposed Seasonality: The budgeting cycle at the firm dictated that initial development could not occur until the end of Q1, because this was the time during which development needs were articulated and chosen at the business level. Further, development usually had to be completed during the fiscal year (usually by the end of Q3), in order for the development to be accounted for internally in that year’s budget. This practice imposed a “Seasonality” on the software cycle. If interruptions occurred in the middle of this window, such as auditing or compliance needs, the schedule could not slip and would have to be accommodated. This was accomplished either with overtime (running an increased risk of errors) or by “shifting” development work into maintenance. Both of these responses, while allowing development to stay on schedule, would increase the maintenance tail.
Unplanned Upgrades: A developed application requires scheduled upgrades, which conceivably could be known and planned for. However, these upgrades are not included in the budget in order to keep “perceived” maintenance costs down. These known but unplanned for upgrades are then treated as “emergency upgrades,” creating an additional resource burden that must be made up for in an ad hoc basis.
All of these explanations showed “hidden” dynamics that were shifting costs to software
maintenance. To understand the impacts of these dynamics, we next developed a series of simulation models that would help quantify their impacts, generate potential solutions, and test options for high‐leverage intervention.
System Dynamics for the IT Portfolio
We utilized the system dynamics modeling (SDM) methodology, which enables analysis that focuses on system performance over its life span. (Sterman 2000) It uses quantitative modeling to capture a firm’s processes and actors, and simulate different scenarios over time. The focus on dynamic behavior enabled us to identify the unexpected or counter‐ intuitive ways in which maintenance process may perform, and to capture the three explanations described above.
The SDM methodology employs the building blocks of group model elicitation (a process for collecting information from multiple sources and perspectives), and causal diagramming (a process for capturing and displaying this information to facilitate understanding) to help evaluate system performance. The information gathered during these steps is then combined with data for key variables, when available, to develop testable simulation models. While these models generate significant statistical data, the main benefit often comes from modeling‐based insights of general applicability that can be applied throughout the organization.
The model is built around the following logic: Business Demand is created by business units for software applications and forms a development backlog. The desired pace of development is dictated by business logic and the market.
Application Development occurs to meet business needs (“drain the backlog”) and is controlled several factors, including the time available and resources available, experience, and fatigue.
Application Review as a quality assurance process to evaluate development and determine next steps. Basic Maintenance occurs once development is completed applications require only “normal” or expected maintenance.
Additional Maintenance is required if, once development, unplanned maintenance is needed to fix problems from development.
Figure 2 shows a system dynamics diagram, referred to as a stock and flow diagram, to capture this logic. (see Abdel‐Hamid 1984 and 1998, and Abdel‐Hamid and Madnick 1988 for core structure.) Development Backlog Business Demand Application Development Application Review (Q&A) Addition or Unnecessary Maintenance Maintenance Discovery Rate Maintenance Completion Rate Basic Maintenance Applications RequiringBasic Maintenance
Figure 2. Model Framework
For reasons of parsimony, we briefly present the core of the model in Figure 2. (Appendix 1 has a full view of the system dynamics model.) The rectangles denote stock, or a collection of tasks (for example, the number of application tasks in the “Development Backlog”), and the arrows with hour glasses denote the flows, or the rates at change between the stocks (for example, the rate at which these tasks are finished through “application development.”)
There are two key dynamic insights to this model. First, the activities controlling the rates of change all pull from the same resources (similar to the logic in the pie chart in Figure 1). If the maintenance stocks rise and require increasing additional maintenance work, it will draw an increasing share of resources and ultimately reduce the ability to drain the development backlog and develop new applications. The organization, however, did not originally recognize this linkage. Business units were empowered to develop applications without considering the long‐term maintenance challenges. By incorporating and connecting the two processes, we were able to show that this “stove piping” could hurt overall performance and limit the business units in ways they did not anticipate.
Second, applications have the ability to get stuck in the “additional maintenance” cycle. For example, unexpected “rework” can be discovered during the maintenance cycle that was previously unknown, creating unplanned for but critical maintenance tasks. What makes this situation worse is its propensity to compound into greater effects. For example, if a large “development” task is shifted into maintenance by rushing development, such as automating a process, the maintenance staff may never have the budget to develop the solution. Instead, they are forced to find incremental “work‐ arounds” in the short‐term, which, while having smaller costs in a specific year, can add up significantly over the applications lifespan.
In sum, the model framework alone shows how the distinctions between development and maintenance can be blurred, challenging the introductory definitions described on page one. Thus, what appear to be small events (such as rushed development) can actually lead to large long‐term effects, particularly when they are compounding.
Below in Figure 3 we present three cases which capture the overall situation at the company by running the model five years into the future. In blue we show the base case, a stylized ideal case in which the Maintenance Backlog is maintained around a flat level (rising and falling in small amounts throughout the year) and New Applications are unimpeded. Next, in the assumed case, shown in green, we simulated using parameters gained from interviews as to how things were going at the firm. While staff knew the Maintenance Backlog was growing year‐over‐year, they assumed it would grow at small percentages, and there would be very little effect, if any, on New Applications.
Maintenance Reveiw Backlog
400 300 200 100 0 0 6 12 18 24 30 36 42 48 54 60 Time (Month) packag es Maintenance Backlog Current Situation Base Case Cumulative Applications 4,000 3,000 2,000 1,000 0 New Applications Base Case 0 6 12 18 24 30 36 42 48 54 60 Time (Month) Current Situation Assumed Case Assumed Case Figure 3: Model Simulation Output
Finally, we simulated the current situation, shown in red, by activating the problematic dynamics we had uncovered. Here we see a far worse picture— the Maintenance Backlog continues to grow at a rapid rate and New Applications are significantly reduced. These results highlighted the compounding effects of small software processes and provided the basis for communicating about necessary improvements to management of the IT Portfolio throughout the company.
Conclusion: Valuing and Managing the IT Portfolio
This work has shown that the evolution of information systems is a dynamic process that links the “physics” of software processes with social and business systems, including key users, developers, and managers throughout organizations. The management of software maintenance is particularly complex because “cause and effect” may be separated in time and space (i.e., the drivers of maintenance costs may arrive from multiple sources and with significant delays). These characteristics make attributions about system improvement difficult, and can impede the ability to manage costs. We found that prior attempts to reduce maintenance budgets were limited because they did not include critical linkages among
different parts of the organization, temporal dynamics (i.e., “solving” a problem today might actually cause more problems later), and experiments done in low‐cost modeling settings prior to expensive real‐world implementation.
We believe that powerful new opportunities exist to use system dynamics modeling to understand the key drivers of maintenance costs and develop high‐ leverage solutions to mitigate expenses. In Figure 4, we propose a feedback system for change.
Evaluate system-wide drivers of maintenance
Identify high-leverage options to reduce total
Validate improvements using models prior to major system changes Implement, measure, and refine Evaluate system-wide drivers of maintenance costs Identify high-leverage options to reduce total
Validate improvements using models prior to major system changes Implement, measure,
Figure 4. Methodology of Investigation
We believe that a holistic approach that begins by evaluating system‐wide drivers of costs, indentifying opportunities to manage costs, testing and validating these approaches in low‐cost simulation models, implementing, measuring, and refining interventions, and then again reviewing the overall system can improve organizational ability to manage and improve the IT Portfolio.
1. IEEE Standard for Software Maintenance. IEEE Std 1219‐1993. Institute of Electrical and Electronics Engineers, inc., New York, NY. 2. Takang, A.A., AND Grubb, P.A.. Software
Maintenance Concepts and Practic. Thompson Computer Press London, UK. 1996
3. Sterman, J., N. Business Dynamics: Systems Thinking and Modeling for a Complex World. Chicago: McGraw‐Hill/Irwin. 2000
4. Abdel‐Hamid, T.K. "The Dynamics of Software Development Project Management: An Inte‐ grative System Dynamics Perspective," un‐ published Ph.D. dissertation, Sloan School of Management, MIT, January, 1984.
5. Abdel‐Hamid, T.K. "The Dynamics of Software Project Staffing: A System Dynamics Based Simulation Approach," IEEE Transactions on Software Engineering, forthcoming 1988. 6. Abdel‐Hamid, T.K. and Madnick, S.E. Software
Project Management, Prentice‐Hall, Inc., En‐ glewood Cliffs, NJ, forthcoming 1988b
Development Backlog Adding New Application Needs Developing Application Review (Q&A) Unexpected or Unnecessary Maintenance Maintenance Discovery Rate Maintenance Completion Rate Basic Maintenance Applications with Basic Maintenance Fraction Discovered To Require Non-Basic Maintenance Actual time Available for Development Actual Development Rate -+
-Desired time For
Development Desired Development Rate -+ Development Schedule Pressue + -+ Experience Fatigue Communication Difficulties -+ + + + + Seasonality Effect -Other Requirements (Compliance) -+ Maintenance Corrective Rate Maintenance Effectiveness + + +
Founded in 1999, the Center for Digital Business is the largest research center in the history of the Sloan School. We are supported entirely by corporate sponsors whom we work with closely in directed research projects. The Center has funded more than 45 Faculty and performed more than 60 research projects. Our mission is to join leading companies, visionary educators, and some of the best students in the world together in inventing and understanding the business value made possible by digital technologies. Our interactions are a dynamic interchange of ideas, analysis, and reflection intended to solve real problems.
Examples of Current Focused Research Projects: •Networks as Platforms (Cloud Computing) • Deriving Competitive Advantage from IT • The Business Implications of Enterprise 2.0 • Productivity and Internal Knowledge Markets • Web Site Morphing to Individual Cognitive Style • Measuring the Productivity of Information
•Improving Hospital Operational Efficiency and
Risk Management with Systems Dynamics
•Using Systems Modeling to Predict, Manage and Improve Software Application
Development and Maintenance
The Center for Digital Business is focused on understanding the impact of technology on business value, and developing tools and frameworks for our sponsors to use for competitive advantage. Our goal, in part, is to reduce that timeline through basic and applied research, engagement with industry sponsors, and the sharing of best practice, and the MIT’s credo of combining rigor with relevance is well served. We are co‐located with MIT Sloan’s Center for Information Systems Research and the Center for Collective Intelligence to facilitate collaboration. Our cross‐campus collaborations include work with the Media Lab, W3C, Computer Science and AI Lab, and Communications Futures Program.
We are organized into four areas of expertise – or Special Interest Groups:
1. Digital Productivity
2. Digital Marketing
3. Digital Services 4. Digital Health
The Center for Digital Business gratefully acknowledges the support of our current Sponsors. Founding Sponsors Cisco Systems CSK Corporation General Motors McKinsey SAP Suruga Bank Research Sponsors BT FT Orange Labs Liberty Mutual Member Sponsors Google HP Oracle SAS Solution Services CONTACT INFORMATION
MIT Center for Digital Business MIT Sloan School of Management 5 Cambridge Center, NE25‐7th Floor Cambridge, MA 02142
Telephone: (617) 253‐7054 Facsimile: (617) 452‐3231
David Verrill, Executive Director Glen L. Urban, Chairman
Erik Brynjolfsson, Director
Carlene Doucette, Executive Assistant Joanne Batziotegos, Financial Assistant Please visit our website for more information: