A Simulation-Based Approach in Support of
Project Management Training for Systems
Engineers
Izack Cohen,1 Michal Iluz,2 and Avraham Shtub1, *
1Industrial Engineering and Management, Technion—Israel Institute of Technology, Haifa, Israel 32000 2Rafael Advanced Defense Systems Ltd., Haifa, Israel 61532
PROJECT MANAGEMENT TRAINING FOR SYSTEMS ENGINEERS
Received 9 October 2011; Revised 25 September 2012; Accepted 25 September 2012, after one or more revisions Published online in Wiley Online Library (wileyonlinelibrary.com).
DOI 10.1002/sys.21248
ABSTRACT
Project management is taught in most systems engineering graduate programs. Project management courses and training aids traditionally focus on aspects of project scope—for instance, scheduling and resource allocation methods—and tend to neglect product scope elements such as system performance. This article offers a simulation-based approach for project management training that integrates project and product aspects within simulation training. The result is an integrated approach that guides the trainee through the initiation, conceptual design, planning and execution phases of the simulated project. The trainee explores the tradeoffs between project and product constraints and demands—just as if he or she were working on a real job. A pilot experiment that we conducted and the trainees’ evaluations of the simulation provide an initial indication that the proposed approach is efficient, motivating us to suggest a framework for its use in systems engineering programs. © 2013 Wiley Periodicals, Inc. Syst Eng 16: 000–, 2013
Key words: project management; systems engineering; simulation based teaching; project team builder
1. INTRODUCTION
While project management courses are common in under-graduate and under-graduate programs, teaching project manage-ment as part of graduate programs in systems engineering is unique because few programs take into account the overlap-ping and linkages between the project management and
sys-tems engineering domains. These links are well documented (see, for example, Sage and Rouse [2009] and Pyster et al. [2012: Part 6]) and highlight the fact that to achieve good performance in a complex project, its systems engineer needs to balance both technical and managerial aspects. For exam-ple, a systems engineer who opts for a novel technical alter-native may win praise for his/her professional performance and for creating an innovative product image. The choice, however, may also lengthen development, leading to higher costs and perhaps to an immature product. Technical issues are typically ascribed to the product domain and managerial ones to the project domain [Sharon et al., 2011].
A literature review indicates that defining curriculum con-tent for systems engineering graduate programs is an interest-ing and widely debated issue [Jain et al., 2007; Squire and Jenkins, 2003; Squires and Larson, 2009; Squires et al., 2010; Fabrycky, 2010]. The continuing efforts to develop the
IN-Contract grant sponsor: Bernard M. Gordon Center for Systems Engineering at the Technion—Israel Institute of Technology.
*Author to whom all correspondence should be addressed (e-mail: [email protected]; [email protected]; [email protected]).
Systems Engineering © 2013 Wiley Periodicals, Inc.
COSE reference curriculum [Jain et al., 2007; Squires et al., 2010] clearly indicate that the systems engineering commu-nity recognizes this need. These efforts focus on defining what needs to be taught—and rightly so, given that this is the essence of curriculum development.
Most graduate systems engineering programs offer at least one project management course. Squires and Cloutier [2010] even list “ project planning system life cycle processes” as one of six levels of course categories in their proposed frame-work.
In this article we focus on how to teach such a course. Current project management courses tend to focus on project scope, which means that systems engineers who graduate from such programs do not explore the symbiosis between project and product scopes and their associated tradeoffs. We suggest that project management training can be improved by allowing the trainee to explore these project/product trade-offs. Specifically, we suggest that Simulation-Based Teaching (SBT) is a suitable approach for achieving this objective. Studies in various areas have shown that SBT is effective [e.g., Ruohomaki, 1995]). Existing project management simulators do not support product/project tradeoffs so we expand the notion of activity modes to model these links and implement them within the Project Team Builder (PTB)—an existing simulator software. Finally, we test the simulator performance in class and determine that it is an efficient tool for training systems engineers in project management.
The paper is organized as follows: In Section 2 we review the relevant literature. In Section 3 we present the model and demonstrate how it is implemented within PTB. Section 4 provides an illustrative example, and Section 5 contains a description of a pilot experiment and its results, followed by a suggestion of how to use SBT in class and managerial insights that are given in Section 6. Section 7 concludes the paper and suggests directions for further research.
2. LITERATURE REVIEW
Our approach integrates two streams of research literature: SBT and project management in the context of systems engi-neering. In this section we briefly review the relevant litera-ture from both streams.
We follow Ruohomaki [1995], who analyzed the SBT tool and its accompanying learning processes, and encourage edu-cators to use it. SBT has proved its effectiveness in practice in fields such as medical training [De Giovanni et al., 2009; Fraser et al., 2009; Park et al., 2007], software engineering training [Pfahl et al., 2001; Ioana et al., 1999], flight training and decision making (improving decision making under stress) [Rolfe and Staples, 1988, Cohen, 2008], general man-agement training [Salas et al., 2009] and project manman-agement training [Shtub, 2004, 2011a, 2011b].
Several specially designed simulators support project management training (e.g., Celemi, Clarrus, Double Masters, Fissure, Forio, Polstar, Prendo, Race to Results SimProject, SimulTrain, SMG, Synergest). Because these simulators fo-cus on project management issues (the project scope), the user is not required to go through the initiation and design phases that translate stakeholders’ requirements into the product
scope. Furthermore, most simulators do not require the trainee to tradeoff system performance against project cost and schedule. In other words, the common approach is that the trainee gets a predefined scenario and has to plan the project (i.e., develop a plan that takes into account the schedule, budget, risk, and resources) and simulate its execution. To the best of our knowledge, simulators that aim to model the links between project and product scopes are only in their initial development stages [e.g., Squires et al., 2012], in contrast to the suggested approach that is already used in academic and industrial settings. To recapitulate, our approach is distinc-tively different than existing ones in that it links the project and product scopes, thus creating an opportunity for the trainee to see and work with the reciprocal influences of the two scopes. This approach is supported by the literature about project management and systems engineering, as we explain next.
The literature reflects a long-standing effort to rationalize the links between project management and systems engineer-ing. The interested reader is referred to a Navy guide [COSC, 1994] that mentions possible conflicts (p. III-13) and needed integration (p. XXII-1) of project management and systems engineering. More recently, Eisner [2011], surveying the links between project management and systems engineering, en-couraged using an integrated product and project scope man-agement approach. Meier [2008] found that inadequate systems engineering and project management integration in the preacquisition phase of large defense and security projects led to significant cost and schedule overruns. Our training focus on the design and planning phases is in agreement with Meier’s conclusion that most unsuccessful projects fail at the beginning due to lack of integration between project manage-ment and systems engineering. Sharon et al. [2011] experi-mented with the suitability of various management tools for an unmanned aerial vehicle project. Their results indicate that project and product scopes are complementary; thus, the experiment’s participants perceived tools dealing with both aspects as more efficient.
The present research draws from the above-mentioned literature to offer an efficient way to combine simulation for a training experience that links project and product scopes. The importance of such linkage cannot be underestimated [Eisner, 2011]. Moreover, it has been shown to be missing in failed projects [Meier, 2008]. The approach suggested here is unique among the current project management teaching ap-proaches.
3. METHODOLOGY
Section 3.1 presents our methodology and modeling approach and introduces the model, its dynamics and assumptions. Section 3.2 elaborates on the basic features of the PTB simu-lation tool that was used in our experiment and how it imple-ments the model.
3.1. The Model
The motivation to reduce the gap between actual project management and systems engineering training in project
management is based on the literature review and on our experience.1 We model a project as a group of tasks charac-terized by durations, costs, resource demands, precedence relationships and uncertainty. Unlike common project man-agement simulation software where the objective is to meet the project scope without considering product scope con-straints, the proposed approach aspires to meet both project and product scopes (e.g., to ensure the final product’s prespe-cified performance level subject to budget and schedule con-straints).
In a project’s design stage, engineers and others involved in the project choose a technology from among different alternatives. These choices affect many project elements such as the final product characteristics (e.g., performance, reli-ability and maintainreli-ability) and the project’s forecasted per-formance (e.g., the probability of meeting the due date, life cycle costs and direct and overhead costs). We model this link between characteristics and performance through the well-known notion of time–cost tradeoff and task modes [e.g., Shtub et al., 2005: 498]. Typically, a mode is defined as a combination of resources assigned to a task that determines its duration and cost; such is the case when, for example, extra resources can be added to shorten task duration. The time– cost tradeoff is closely linked to the critical path method (CPM), which exploits the tradeoff so as to shorten project duration in a cost effective way (see an early description of the time–cost tradeoff in Archibald and Villoria [1967: 120]). When the task can be performed in only one of several discrete modes, then the problem is called the discrete time–cost tradeoff scheduling problem (DTCTSP), which has no com-putationally efficient algorithm to find its optimal solution (it is defined as an NP-hard problem). Thus, heuristic solution methods are used to solve it.
The stochastic version of DTCTSP (SDTCTSP), which we use, is closer to reality since the durations and costs of discrete modes are subject to uncertainty. We follow the approach taken by Cohen et al. [2007] for modeling uncertainty in the continuous version of the problem and apply it to SDTCTSP, about which the research literature has almost nothing to say, as Herroelen and Leus [2005] noted. We find it natural to interpret each discrete mode as a technological alternative with its estimated performance, cost, schedule, and uncer-tainty. Uncertainty here is captured through a distribution of the corresponding parameters for each mode. When simulat-ing a project, these parameters’ realizations (e.g., task dura-tion) are determined according to their corresponding distribution.
We build on the modes to describe real-life tradeoffs. Assume a classical tradeoff situation between two alterna-tives—one is riskier, with high-performance product poten-tial and the second uses mature, moderate performance technology. This type of situation is easily modeled in the PTB through two modes: choosing the first option produces a high performance level (benefit) at a price of larger uncer-tainty in duration and cost, and the second mode results in a lower performance level but with more secure schedule and cost. The trainee selects a mode by weighing up technical considerations and utilizing available project analysis tools (e.g., life cycle cost analysis, decision tree modeling, etc.). Figure 1 schematically presents a trainee’s typical initial project planning process. Section 3.2 provides a detailed account of the implementation and use of our model within the simulator.
3.2. Implementing the Model
Project Management Trainer (PMT) is a simulation training tool for project management that the Technion’s graduate systems engineering program has used for several years. PMT comes with a set of predefined scenarios (a scenario library).
1All the authors participated, managed, and directed aerospace and defense
R&D projects and consulted to leading organizations in project management and systems engineering fields.
Figure 1. A schematic presentation of the SBT working process.
Once a scenario is selected, the trainee has to plan the project and execute it in a stochastic, dynamic environment. A num-ber of experiments performed with the PMT [Davidovitch et al., 2006, 2008] revealed increased learning process effi-ciency and effectiveness. At the same time, though, students had difficulties integrating the project management body of knowledge and their simulation experience with the tools and techniques taught in the systems engineering curriculum— exactly the gap that we aim to reduce, and which led to our introduction of the model, described in Section 3.1, into the PMT platform. To tackle this gap a new simulator was devel-oped—PTB. We now describe PTB’s primary features, with the focus on linking product and project scopes.
PTB can simulate both single projects and programs or portfolios. The simulation is controlled through a simple user interface, and no knowledge of simulation or simulation languages is required. To capture the diversity in project types that systems engineers encounter, PTB uses case studies (depicted as scenarios). Each case study is a project or a collection of projects performed under schedule, budget, and resource constraints in a dynamic, stochastic environment (e.g., a portfolio of projects, which has to be completed, sharing the same pool of scarce resources and common cash flow). The details of these case studies are built into the simulation, and all the data required for analysis and decision-making are easily accessed through the user interface. A user-friendly case-study generator facilitates the development of new case studies as required by the trainee or trainer.
The case studies are dynamic in the sense that the situation changes over time. A random effect is introduced to simulate environmental uncertainty and, as in reality, decisions made by the user (e.g., changing a technological option, hiring or assigning workers) impact the state of the simulated system. PTB incorporates a model-based approach through a built-in decision support system. This system is based on the best practices (tools and techniques) listed in the Project Manage-ment Body of Knowledge [PMI, 2008]. The freely accessible tools and techniques support scheduling, budgeting, resource management, monitoring and control. To further support de-cision-making, PTB has a built-in database that also tracks the current state of the simulated system.
PTB was designed as a user friendly teaching and training tool. Although quite complicated scenarios are simulated and the decision support tools are sophisticated, a typical user can learn how to use PTB in less than an hour (Fig. 2 presents GUI examples).
Unlike traditional SBT tools for project management train-ing, the PTB approach separates the simulation engine from the scenario library. This division leads to high flexibility, provided by a scenario building program with which the user (a trainee or its trainer) can build project scenarios based on real or imaginary projects. Each scenario can focus on one or more aspects of project management (system requirements, scheduling, resource management, budgeting, risk manage-ment, monitoring and control, etc.). The trainer using this tool can build any number of scenarios based on the training objectives and the required learning outcomes. Furthermore, trainees can study a real project and build a scenario based on the study results.
The concept of modes, described in Section 3.1, serves to link the project and product scopes in a structured way. Each task may have single or multiple modes. Each mode impacts one or more project attributes including product scope, project scope, task (distribution) duration, the cost of performing tasks, and the resources required. Consider, for example, the development of a new radar system. An important component in a radar system is the transmitter. Suppose that two design options (modes) exist: (1) Modify an existing transmitter or (2) design a new transmitter from scratch. Each option may have a different impact on a task’s performance, duration, cost, risk and required resources. Furthermore, each “ techno-logical mode” may be executed in one or more “ operational modes” ; for instance, the operational modes may be a team consisting of one engineer and one technician or a subcon-tractor modifying an existing transmitter. Mode selection decisions affect the duration, cost, and risk (as well as other performance measures). The scenario builder is used to define the technological and operational modes for each project task (some tasks may have only one possible mode). The trainee simulating the project using the simulation engine must select a mode for each task, taking into account the organizational strategy (e.g., design to cost vs. cost–benefit maximization) as well as stakeholders’ business and technical needs, with the goal of providing a quality product that meets these needs. Note that mode selection for one task may depend on the modes selected for other tasks; for example, the radar range depends not only on the transmitter power but also on the gain of the antenna as well as on other parameters.
Thus, the first step in transforming PTB into a simulator that supports project management training for systems engi-neers includes introducing modes as part of the scenario information. By doing this, the project planning process is extended from mere scheduling and resource allocation (which is supported by most project management simulators) to the selection of task modes. The selection determines the performance of the task-related system component (e.g., the output power of the radar transmitter) along with the task duration distribution (e.g., according to a beta distribution with minimum duration of 1 week, most likely duration of 2 weeks, and pessimistic duration of 4 weeks), cost, and re-source requirements.
The next step introduces a functional relationship between the decisions regarding the individual project tasks and the overall system performance, which is done by modeling the problem through formulas that estimate the system perform-ance. Returning to the radar example, after the trainee selects alternatives that define the system parameters (e.g., antenna gain and transmitter power), the software uses relevant for-mulas (e.g., the radar equation; see Skolnik [1970: 1–6]) to calculate the system performance (e.g., the radar’s range of detection for a given target) and the conformance to predeter-mined customer requirements and constraints. Each perform-ance measure may be given a score that estimates the benefit gained from the product scope aspect. The project scope performance measures (such as overall cost and project com-pletion date) are also calculated, giving a complete picture of the project performance.
Figure 2. GUI examples of a network model view and a Gantt view with details of a specific task. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]
4. AN ILLUSTRATIVE EXAMPLE
We now present an illustrative example to demonstrate the process that a trainee goes through when using PTB, focusing on the link between the project and product scopes.
We use an example derived from a real-world project whose objective is to develop a replacement communication system for airborne platforms and ground stations. The pro-ject is performed in a matrix organization—meaning, re-sources are shared by several projects. Thousands of these new systems are expected to be produced and sold over the
next 5 years. The specifics of the project include development of a transceiver conveying a sound signal (speech) over a wireless communication channel. The main subsystems are analog and digital units, a power amplifier (responsible for amplifying transmission signals), a power supply subsystem, and an antenna. A typical engineering tradeoff in such com-munication systems, with which we want PTB users to grap-ple, is whether to improve the receiver reception sensitivity or to increase the transmission power.
In the first step, either the trainer or trainee (the latter, in advanced courses) uses the scenario building program to set
up the example’s scenario. The trainee can insert customer requirements, which are detailed in the technical specifica-tions (e.g., minimal signal-to-noise ratio of 4 dB). The trainee can also input the system design requirements that comply with customer requirements (e.g., minimum detected signal and range attenuation according to flight distance, maximum physical volume constraint). Various formulas enable the trainee to calculate the compliance between a suggested de-sign and customer requirements (e.g., transmission power— attenuation must be larger than the minimum detected signal). At this point the trainee can decide on various design parameters and consider their respective tradeoffs. For exam-ple, doubling the transmission power provides an additional 3 dB, but in order to do so a stronger power supply is needed, as well as improved thermal dissipation. Implementing these changes may extend mechanical design duration, possibly even necessitating a whole new design process. On the other hand, when considering the reception alternative, the same 3 dB can be gained if a more sensitive receiver is developed. The alternatives and their related project activities are sum-marized in Table I. The simulator takes the formulas and
automatically calculates the resulting performance with re-spect to the trainee’s choices (see Fig. 3 for an illustration of a screen print).
As in reality, first the trainee needs to go through the design stages and analyze the technical alternatives and their impact on performance, schedule, and cost. Once an alternative is selected, its execution may be expedited, for example, by recruiting more resources at an additional cost.
PTB supports presentation of the project in a Gantt chart view. The Gantt chart is updated according to the selected alternative. Figure 4, for example, presents a list of tasks with their precedence relations and durations. A critical path analy-sis based on the most likely durations reveals that tasks 1, 3, 5, 9, 10, and 12 are critical—that is, changing their duration changes the project’s duration.
At this stage the trainee has to make “ systems engineer-ing” decisions regarding the technical alternatives and at the same time consider project scope implications (e.g., on cost and schedule). Such a process is holistic in that it examines all the project elements and fills in the missing link between project scope and product scope.
Figure 3. PTB automatic calculation of performance according to relevant formulas inserted by the scenario builder. [Color figure can be
viewed in the online issue, which is available at wileyonlinelibrary.com.]
Figure 4. A Gantt chart view of the project. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]
Once the planning is finished, the project is simulated. During the simulation the trainee can intervene and change his/her choices (e.g., change a performance mode of an activ-ity, recruit more resources, etc.). At the end of every simula-tion run, PTB presents the following indices: system performance measures that were calculated according to the trainee’s choices, project cost and duration, and simulator run time. We note that this illustrative example is only one of numerous possible scenarios and technical disciplines that can be exercised using SBT. SBT facilitates an integrative, controlled environment for the training experience in both systems engineering and project management tools. More-over, as long as the scenario is coherent, the trainee will understand the links and mutual interactions between the product and project scopes.
5. THE EXPERIMENT
The simulator was tested in order to verify that it is a good project management training tool for systems engineers. Al-though this is a pilot experiment that we intend to extend in the future as more data are gathered, it is important as it provides us with an initial indication of the importance and effectiveness of the SBT approach in the context of systems engineering and project management.
To get as broad an evaluation as possible, we had three groups try the simulation: (1) 16 highly experienced systems engineering project managers (ages 48–65, with an average age of 57); (2) 17 experienced systems engineering project managers (ages 28–62, average age of 46); and (3) 18 students in the Technion’s graduate systems engineering program without project management experience (ages 29–47, with an average age of 37). Figure 5 provides the project management experience distribution for the first two groups.
The participants simulated a scenario based on a real systems engineering project, as described in Section 4. They analyzed customer requirements and executed the project while having to meet customer requirements (e.g., meeting a minimum benefit constraint) with the objective of minimizing the project’s cost.
Following the simulation, we asked the trainees,2 i.e., the groups’ members: Is the simulator a useful systems engineer-ing teachengineer-ing aid in academic courses? Their answers to this question reflect their high evaluations of the simulator as a teaching aid. The experienced and highly experienced groups’ evaluations were higher: mean score of 4.18 and 4.06 on a scale of 1 (low) to 5 (high), respectively. The students’ score was 3.88. The respective coefficients of variation (i.e., the standard deviation divided by the mean), 0.19, 0.19, and 0.33, testify that variability in evaluations was rather low, especially among the experienced and highly experienced engineers. To make sure that the evaluations were not biased by the performance, hypotheses that higher scores (in either benefit or cost) lead to higher evaluations were checked and rejected.
The experiment results indicate that trainees who partici-pate in SBT using the PTB tool find SBT to be useful in teaching project management to systems engineers. We think that experienced system engineers are better qualified to judge the usefulness of the simulator as a training tool than inexperienced students. Thus, although all groups gave the tool high scores, the higher (even if not significantly higher) evaluations by the more experienced engineers are reassuring.
6. THE NEXT STEP—A FRAMEWORK FOR IN-CLASS PROJECT MANAGEMENT SBT
Although we draw from our experience and lessons learned with PTB, the suggested teaching framework suits any other simulation tool that is developed to combine project and product scope linkages.
At present the PTB scenario builder and simulator engine are used as part of the graduate project management course for systems engineering students at the Technion. The course combines lectures and simulation exercises.
Figure 5. Systems engineering project management experience distribution among the (a) experienced group and the (b) highly experienced
group. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]
2We also measured and analyzed trainees’ performance (e.g., achieved cost,
benefit, benefit–cost ratio, run time, etc.) but omit these details as they do not contribute directly to the focus of this article.
The first part of the course is based on predefined scenar-ios, each focusing on a specific subject. The scenarios start with deterministic unconstrained projects and gradually move to resource constrained scenarios, budget constrained scenar-ios, and, finally, to stochastic, multiple constrained scenarios. In the second part of the course students develop scenarios based on real projects. They are given a problem to solve and then, as teams, use the scenario builder to develop a scenario that includes the project data, focusing on operational and technological alternatives. Each team analyzes the scenario and develops a project plan. They simulate the plan several times and present the results (averages and variances) of the schedule, budget, and system performances. The solutions are discussed in class, with the focus on the pros and cons of the different solutions.
At the end of the course, the discussion focuses on the lessons learned from the scenario development and from the development and execution of the project plans.
7. SUMMARY AND CONCLUSIONS
The complex interrelated issues of systems engineering make development of a curriculum for graduate studies in systems engineering a widely debated subject. Most discussions focus on what should be taught. This paper deals with another aspect of curriculum development: how we should teach and what tools and techniques can be used to enhance the efficiency and effectiveness of our teaching—specifically in regard to pro-ject management teaching.
Based on our experience from developing and using a Simulation-Based Training environment for teaching project management to systems engineering students, we created a simulator engine and scenario builder. This software uses the concept of modes to train students to handle the entire life cycle of a project—from the concept phase to the planning phase, and, finally, to the execution phase. The modes repre-sent technological and operational alternatives and realisti-cally reflect the impact of favoring one alternative over another on project and product scopes.
By separating the scenario builder from the simulation engine, trainers can develop an infinite variety of scenarios early on in the course, and, later in the advanced parts of the course, students can do the same. Developing and simulating scenarios of real (or imaginary) projects in the final stages of the course ensure that students integrate all the different issues covered by the course. A good course in project management for systems engineering graduate students combines lectures in the body of knowledge and specific scenarios that are matched to each lecture and focus on the main issues being discussed. The SBT approach using the PTB tool was highly evaluated by users—inexperienced systems engineering stu-dents and systems engineers with a range of experience levels. The main contribution of this research is the creation of a holistic approach that compels trainees to apply, interactively, systems engineering, and project management tools in one environment (project). After running the scenario, the trainees understand the links and mutual interactions between product and project scopes, which will hopefully prepare them better to manage real life projects.
This research can be extended in several directions. One option is to focus on how to build scenarios that enhance learning in the most effective way. Alternatively, future re-search efforts can explore the effectiveness of simulators in supporting decisions for real-life system engineering projects and the integration of simulators with other common tools. Work on these research issues is on-going, and, as more data are gathered, we shall perform experiments to test the simu-lator’s effectiveness for the suggested uses.
ACKNOWLEDGMENT
This research was supported by the Bernard M. Gordon Center for Systems Engineering at the Technion—Israel In-stitute of Technology.
REFERENCES
R.D. Archibald and R.L. Villoria, Network-based management sys-tems (PERT/CPM), Wiley, New York, 1967.
I. Cohen, Improving time-critical decision making in life-threaten-ing situations: Observations and insights, Decision Anal 5(2) (2008), 100–110.
I. Cohen, B. Golany, and A. Shtub, The stochastic time–cost tradeoff problem: A robust optimization approach, Networks 49(2) (2007), 175–188.
COSC, Project management and systems engineering guide, Tech-nical Document 108, Naval Command, Control and Ocean Sur-v e il la n c e C en t er, Wash ington , DC, 1 994, http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=G etTRDoc.pdf&AD=ADA281404.
L. Davidovitch, A. Parush, and A. Shtub, Simulation-based learning in engineering education: Performance and transfer in learning project management, J Eng Educ (2006), 289–299.
L. Davidovitch, A. Parush, and A. Shtub, Simulation-based learning: The learning-forgetting-relearning process and impact of learn-ing history, Comput Educ 50(3) (2008), 866–880.
D. De Giovanni, T. Roberts, and G. Norman, Relative effectiveness of high- versus low-fidelity simulation in learning heart sounds, Med Educ 43(7) (2009), 661–668.
H. Eisner, Essentials of project and systems engineering manage-ment, Wiley, Hoboken, NJ, 2011.
W.J. Fabrycky, Systems engineering: Its emerging academic and professional attributes, Proc 2010 Amer Soc Eng Educ (ASEE) Annu Conf Expo, Louisville, KY, June 20–23, 2010.
K. Fraser, A. Peets, I. Walker, J. Tworek, M. Paget, B. Wright, and K. McLaughlin, The effect of simulator training on clinical skills acquisition, retention and transfer, Med Educ 43(8) (2009), 784– 789.
W. Herroelen and R. Leus, Project scheduling under uncertainty: Survey and research potentials, Eur J Oper Res 165 (2005), 289–306.
R. Ioana, J. Collofello, and P. Lakey, Software process simulation for reliability management, J Syst Softw 46(2/3) (1999), 173– 182.
R. Jain, A. Squires, D. Verma, and A. Chandrasekaran, A reference curriculum for a graduate program in systems engineering, IN-COSE Insight 10(3) (July 2007), 9–11.
S.R. Meier, Best project management and systems engineering practices in the preacquisition phase for federal intelligence and defense agencies, Project Management J 39(1) (2008), 59–71. J. Park, H. MacRae, L.J. Musselman, P. Rossos, S.J. Hamstra, S.
Wolam, and R.K. Reznick, Randomized controlled trial of virtual reality simulator training: Transfer to live patients, Amer J Surg 194(2) (2007), 205–211.
D. Pfahl, M. Klemm, and G. Ruhe, A CBT module with integrated simulation component for software project management educa-tion and training, J Syst Softw 59(3) (2001), 283–298.
PMI, A guide to the project management body of knowledge (PMBOK guide), Newtown Square, Pennsylvania, 2008. A. Pyster, D. Olwell, J. Anthony, S. Enck, N. Hutchison, and A.
Squires (Editors), Systems engineering and management. A guide to the Systems Engineering Body of Knowledge (SEBoK), version 1.0, Stevens Institute of Technology, Hoboken, NJ, 2012, www.sebokwiki.org/index.php/Systems_Engineering_and_M anagement.
J.M. Rolfe, and K.J. Staples, Flight simulation, Cambridge Univer-sity Press, Cambridge, 1988.
V. Ruohomaki, “ Viewpoints on learning and education with simu-lation games,” in J. Riis (Editor), Simusimu-lation Games and Learn-ing in Production Management, Chapman & Hall, London, 1995, pp. 13–25.
A.P. Sage and W. Rouse, Handbook of systems engineering and management, Wiley, Hoboken, NJ, 2009.
E. Salas, J.I. Wildman, and R.F. Piccolo, Using simulation-based training (SBT) to enhance management education, Acad Man-agement Learn Educ 8(4) (2009), 559–573.
A. Sharon, O.L. de Weck, and D. Dori, Project management vs. systems engineering management: A practitioners’ view on
in-tegrating the project and product domains, Syst Eng 14(4) (2011), 427–440.
A. Shtub, PMT—the project management trainer, Ninth Int Conf Project Management Scheduling, PMS, 2004, pp. 430–433. A. Shtub, Project management and simulation based
training—re-search and practice, MSK Hesehv, 2011a.
A. Shtub, Project management and simulation with PTB the Project Team Builder, Springer, New York, 2011b, forthcoming. A. Shtub, J.F. Bard, and S. Globerson, Project
management—proc-esses, methodologies, and economics, Prentice Hall, Upper Sad-dle River, NJ, 2005.
M.I. Skolnik, Radar handbook, McGraw-Hill, New York, 1970. K. Squire and H. Jenkins, Harnessing the power of games in
educa-tion, Insight 3(1) (2003), 5–33.
A. Squires and R. Cloutier, Evolving the INCOSE reference curricu-lum for a graduate program in systems engineering, Syst Eng 13(4) (2010), 381–388.
A. Squires and W. Larson, Improving systems engineering curricu-lum using a competency-based assessment approach, Spec Iss Syst Eng Educ Int J Intell Defense Support Syst (IJIDSS) 2(3) (2009), 184–201.
A. Squires, W. Larson, and B. Sauser, Mapping space-based systems engineering curriculum to government-industry vetted compe-tencies for improved organizational performance, Syst Eng 13(3) (2010), 246–260.
A. Squires, A. Pyster, D. Olwell, S. Few, and D. Gelosh, Announcing BKCASE: Body of Knowledge and Curriculum to Advance Systems Engineering, INCOSE Insight 12(4) (December 2009), 69–70.
A. Squires, J. Wade, B. Watson, D. Bodner, R. Reilly, and P. Dominick, Year one of the systems engineering experience ac-celerator, CSER Conf, 2012, pp. 267–272.
Izack Cohen is a visiting lecturer in the Industrial Engineering and Management Faculty at the Technion. Izack received his B.Sc in Chemical Engineering (1991), his M.Sc in Materials Engineering (1997), and his Ph.D in Industrial Engineering and Management (2004). He managed various technological, logistics, and information systems organi-zations and numerous technological projects. His research interests include: project management, supply chain management, service engineering for healthcare, and innovative maintenance methods.
Michal Iluz received her B.Sc. in Industrial Engineering and Management and M.E. in Systems Engineering in 2001 and 2008, respectively from the Technion—Israel Institute of Technology, Haifa, Israel, where she is currently working towards a Ph.D. degree at the Industrial Engineering and Management Faculty. Her current research interests include simulator-based decision making, shared understanding, and training.
Avraham Shtub is the Stephen and Sharon Seiden Professor of Project Management at the Technion—Israel Institute of Technology, Haifa, Israel. He is the recipient of the Institute of Industrial Engineering 1995 “ Book of the Year Award” for Project Management: Engineering, Technology and Implementation (coauthored with Jonathan Bard and Shlomo Globerson) (Prentice Hall, 1994). He is the recipient of the Production Operations Management Society 2000 Wick Skinner Teaching Innovation Achievements Award for his book Enterprise Resource Planning (ERP): The Dynamics
of Operations Management. Professor Shtub is the recipient of the 2008 Project Management Institute Professional
Development Product of the Year Award for the training simulator “ Project Team Builder—PTB.” His books on Project Management were published in English, Hebrew, Greek, and Chinese. Professor Shtub was a Department Editor for IIE
Transactions, he was on the Editorial Boards of the Project Management Journal, The International Journal of Project
Management, IIE Transactions, and the International Journal of Production Research.
