Improving Software Project Management Skills
Using a Software Project Simulator
Derek Merrill and James S. Collofello
Department of Computer Science and Engineering
Arizona State University
Tempe, AZ 85287-5406 USA
collofello@asu.edu
http://www.eas.asu.edu/~sdm
Abstract
Software project management skills are becoming an important component of software engineering education. Software engineers working in teams need to carefully plan and coordinate their efforts in order to be successful. Unfortunately, most universities provide inadequate education in software project management. Most use lecture-based approaches which provide the necessary steps in software project management, but are deficient in providing the students with hands-on experience. Software project simulation provides a bridge between course-based and hands-on experience. It provides an interactive environment of repeatable exercises. It also provides a medium for measurable evaluation of student performance which can be used to customize the education process to fit the needs of individual students.
Our work focuses on using system dynamics modeling for simulating software development activities because of its ability to dynamically represent relevant project attributes in the software development process. A system dynamics model of the incremental software development process has been developed and validated. This paper describes a process for utilizing this system dynamics model to create simulation environments suitable for addressing specific education objectives. Benefits and guidelines for use of a tool of this kind are provided. Our experience with using this tool in a large class is also described.
1: Background
1.1: Why Focus on Software Project Management?
There is a clear need for effective software project management. The “software crisis”, the long proclaimed ineffectiveness of software development projects to maintain their schedule, cost, and quality, continues to plague most development projects [3][5]. However, many of these projects may not have been a failure if certain common project management pitfalls had been avoided. Barry Boehm notes in [3], “Poor management can increase
software costs more rapidly than any other factor.” Moreover, both Capers Jones and Joyce Statz note inadequate project management training and inadequate standards with which to judge project management performance as the root causes of poor project management [5][8]. A standard for judging project management performance is needed because it is difficult to improve an activity without a method for evaluating that activity’s performance.
1.2: How Can Software Project Management Training Be Improved?
Software project management involves, among other tasks: planning, tracking, and control of a software development project. These abilities require two things: knowledge, such as knowing common project management pitfalls, and skills, such as the ability to recognize those pitfalls and formulate a new project plan. The methods by which software project management skills are improved in an academic environment are summarized as:
• Texts and Journals, • Facilitated Video, • Lecture and Discussion, • Computer Simulation, • Model Project, and
• a combination of these, such as a typical software course consisting of reading, lecture, and a model project.
These different methods can be compared in terms of their ability to educate (i.e. each method’s effectiveness) and their practical costs (i.e. each method’s efficiency). The motivation for exploring computer simulation as a software
project management training vehicle is best introduced by the following analogy.
Software project management is a highly complex, on-going process similar to flying a plane in rough conditions. A pilot must plan the course, continually track progress, monitor the condition of the plane, and react to problems as they are encountered. Pilots are often trained in flight simulators which provide the pilot with experience in many different scenarios without the high risk and expense of learning using a real plane. A pilot could not be expected to learn piloting skills from a text or instructor alone, because a pilot needs hands-on, active experience to become effective. Similarly, hands-on, active experience is needed to build effective project management skills. The basis for this flight simulation analogy was given in [7] which was built upon the system dynamics modeling of the software development process documented in [1].
An analysis of the advantages and effort required to apply the computer simulation approach to software project management training is given in Section 4. The next section introduces system dynamics models which are used as the simulation engine of this research effort.
2: System Dynamics Modeling
The system dynamics approach to modeling complex systems was first introduced in the late 1950’s at M.I.T.’s Sloan School of Management [4]. The first widely recognized application of this modeling approach to software development processes was presented in [1]. Many other modeling approaches are applicable to modeling software development processes; however, system dynamics modeling provides the ability to capture all relevant attributes of the software development process, product, and personnel which relate to the planning, tracking, and control involved in software project management.
In any modeling approach, the boundaries of what is to be modeled must be defined. For software project management training, the model must represent:
• process attributes such as the number and overlap of development increments,
• product attributes such as product size and complexity, and
• personnel attributes such as engineer productivity and schedule pressure.
The value of these attributes can change over time based on the cause-effect and quantitative relationships between them -- such as the relationship between schedule pressure and engineer productivity. The system becomes
very complex as these relationships connect in cycles to form feedback dynamics. These feedback dynamics are both the power of the system dynamics approach and the reason software project management is very difficult [2]. An example system dynamics model is given in Figure 1. The model is designed graphically with each icon storing the value of an attribute of the system that changes over time. An example system dynamics simulation tool user interface is given in Figure 2.
Depending upon the purpose of the model, some of the attributes and relationships may be left hidden in the user interface. If the purpose is to learn software development processes, the structure of the model and its inner relationships will be open for the user to modify. If the
Figure 2. An example user interface to a system dynamics simulation tool for understanding the inspection process.
Undiscovered Code Errors
quality of code
test exec done
Detected Code Errors error detection rate
test thoroughness test rate
Reworked Test Errors code rework rate
~ test productivity
rework prod
Test Size Completed Test
test rate percentage mp for test Escaped Errors
escape rate
daily mp
test rework done test done
Unit Test error detection and correction
Unit Test Execution Test Completion
Unit Test
Figure 1. An example system dynamics model of the unit testing process.
purpose is to learn the management of a given process, without questioning its structure, the model will be closed and a user interface will abstract the structure of the model for the user. Since the goal here is training in project management, the model structure is hidden from the user. Guidelines for creating and validating system dynamics models can found in [9] and [6].
3: A Process For Developing System Dynamics
Simulation Training
The process, given in Table 1, for creating a simulation environment is followed in order to be certain the training objectives are satisfied. The steps are iterated in order to capture more training objectives each loop around.
4: Guidelines For Use
The decision to use system dynamics “flight simulation” for training is based upon its effectiveness and efficiency. The effectiveness of this approach is divided between the benefits given in Table 2 and the limitations listed below.
• This approach is only applicable to training objectives that focus upon the planning, tracking, and control of development. Other software project management skills (e.g. communication
skills) are better learned using other methods. • Also, this approach requires clearly defined
training objectives. If the objectives are not clear, the training is less likely to be effective and it will be difficult to fit the training into the framework of the rest of the course.
The efficiency of this approach is limited by the one time effort of creating the training tool; however, the tool can easily be recalibrated for different scenarios and reused by any number of students.
5: Application of the Approach
To evaluate the usefulness of simulation in software project management training, we performed a case study in a graduate level software project management course at Arizona State University. The participants were all graduate students some of whom had industry experience. The objective of the case study was to evaluate whether the use of simulation helps students better learn software project management lessons.
The students were surveyed as to their project management background and a “snapshot” of their understanding of the dynamics of software project management was taken by having them describe the direction and strength of relationships in an abstract system
Table 1. An iterative process for developing a system dynamics simulation training tool.
Objective Task Summary
1. Lesson Plan • Decompose the training objectives into topical subunits. • Create a lesson plan for each subunit.
• Determine if simulation is the appropriate approach for each lesson plan. 2. Simulation
Interface
• Determine the relevant process, product, and personnel attributes that will be displayed to the user.
• Determine which attributes are controllable by the user. 3. Simulation
Engine
• Determine the model boundaries • Select or create an appropriate model. • Add any lesson specific model components.
• Automate all parts of the model that are not relevant to the lesson. 4. Simulation
Scenarios
• Specify values for the scenarios (e.g. the number of engineers) • Determine thresholds for judging student performance. 5. Validation • Verify the model’s behavior
• Verify that the scenarios are realistic
• Verify that the thresholds for student performance are satisfactory with respect to the training objectives given in step 1.
dynamics model. After the survey, student teams were formed (including students from industry taking the course via real-time televised broadcasts). Each team was given the role of managing a simulated project. The project was simulated on a training tool built upon a validated, comprehensive model of the concurrent incremental development process created by [9].
All teams were introduced to the details of the project and given historical metrics from past projects. The project was a two increment development project estimated at 40,000 assembly equivalent lines of code. Details of the development process, product, and personnel, including a project plan based upon COCOMO cost estimation data, was provided to the students. The first increment was run without student control and ends on time, within budget, and at the required quality standards. This allowed the students to become familiar with the simulation tool and the simulated project. The student teams were then presented the problem of replanning the second increment with respect to a new 10% increase in product requirements. The controls available to the student teams given at the bottom of Figure 3 are:
• increase mandatory worker overtime,
• hire more inexperienced or experienced engineers,
• modify the amount of engineer effort allocated to quality assurance activities,
• extend the scheduled completion date, and • cut the product functionality by up to 10%.
The teams choose their strategy based upon the following guidelines:
• the project is considered a failure if it does not
have a shorter cycle time, fewer defects, and lower cost than the same project developed with no changes in the project plan; and
• successful projects are judged by their cycle time, quality (consisting of the number of delivered defects and the delivered product’s functionality), and cost (consisting of the number of inexperienced and experienced engineer days multiplied by their costs).
The student teams then defined their plan and documented their expected results. In class, several student teams’ strategies were introduced then played out on the simulator. The whole class watched the result of the teams’ development plan unfold as six months of the development project played out in a matter of minutes. The students could graphically see the changing value of discovered defects, engineering productivity, etc. The results were then discussed and compared to the performance criteria, the team’s expected results, and other teams’ results. The students were then surveyed again and asked their opinions of the exercise.
Of the twelve teams that participated, three performed better than the unmodified project plan in cycle time, quality, and cost. All teams performed better in one of the three areas. The most successful strategies involved some combination of: pushing back the scheduled completion date to handle the new development work, increasing the number of experienced engineers on the project, and forcing worker overtime. The least successful strategies involved adding inexperienced engineers who introduced more defects during development and thus more unexpected rework.
As a result of the exercise, the students gained some insight into the dynamics of a large development project as evidenced by the difference between their initial and final surveys.
Table 2. The potential benefits of system dynamics flight simulation.
Attribute Benefit
Realism The results of the student’s decisions are animated.
Duration Students can immediately compare the simulated project’s results with what they had planned. Extensibility Any project can be simulated and one model can easily run multiple scenarios.
Repeatability A scenario can be repeated until it is mastered.
Measurability The simulator can assess a student’s abilities in order to tailor the training to the student. Also, the simulator can evaluate each student’s abilities to verify the training objectives were satisfied.
6: Future Work
We are continuing to expand and refine the capabilities of our software project management simulator for use in both academia and industry. In academia our target audience of users consists of graduate students in a project management course and undergraduates learning basic software project management in an introductory software engineering course. In industry we are teaming up with a local software development organization to provide support for their project management training. We are also teaming with a management professor to provide higher level software project management training to non-technical project
managers.
We are also in the process of creating custom interfaces to our simulator rather than utilizing the standard interfaces provided by system dynamics modeling software.
Expansions of our modeling capabilities are planned to reflect the key software engineering practices reflected in the Software Engineering Institute Capability Maturity Model. In particular we will be expanding our simulator to reflect software risk management as well as cost of quality issues.
References
[1] Abdel-Hamid, T., Madnick S., Software Project Dynamics An Integrated Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1991.
[2] Abdel-Hamid, T., "Thinking in Circles," American Programmer, May 1993, pp. 3-9.
[3] Boehm, B., Software Engineering Economics, Prentice-Hall, Englewood Cliffs, New Jersey, 1981. [4] Forrester, J., Industrial Dynamics, The M.I.T. Press,
Cambridge, MA, 1961.
[5] Jones, C., Applied Software Measurement: Assuring Productivity and Quality, McGraw-Hill, New York, 1991.
[6] Richardson, G., Pugh, A., Introduction to System
Dynamics Modeling with DYNAMO, The M.I.T. Press, Cambridge, MA, 1981.
[7] Rubin, H., Johnson, M., Yourdon, E., "With the SEI as My Copilot Using Software Process 'Flight Simulation' to Predict the Impact of Improvements in Process Maturity," American Programmer, September 1994, pp. 50-57.
[8] Statz, J., "Training Effective Project Managers,"
American Programmer, June 1994, pp. 43-48.
[9] Tvedt, J., “An Extensible Model for Evaluating the Impact of Process Improvements on Software Development Cycle Time”, Ph.D. Dissertation, Arizona State University, Tempe, Arizona, 1996.
