Educating Software Engineers to Become Systems Engineers

Educating Software Engineers to Become Systems Engineers

Supannika Koolmanojwong and Barry Boehm

Center for Systems and Software Engineering (CSSE)

University of Southern California (USC)

Los Angeles, CA, 90089-0781, USA

{koolmano, boehm}@usc.edu

Abstract

Our two-semester USC core software engineering project course CS577ab devotes its first semester to having students learn and do systems engineering on a real-client project. This requires a good deal of just-in-time lectures, tutorials, and homework to prepare the students, and feedback in terms of mentoring, artifact grading, and live milestone reviews to help them succeed. This paper provides some initial motivation and context; discusses our approach to introduce systems engineering into software engineering relative to that in the GSwE 2009 curriculum guidelines, SEBOK draft 2010, and SWEBOK 2004; describes the course practices during the systems engineering and software engineering semesters; and summarizes the project results and conclusions.

1. Introduction

The increasing diversity and interdisciplinarity of software development raises an awareness that software engineers need to deal with more than just software, they need to participate in engineering the system, too. In the past, a software engineer could just sit on the side in an architecture-specification meeting and wait for a hardware engineer or a systems engineer to give them precise specifications to code up. This frequently leads to awkward software constraints, and major risks or disasters in software system development [2]. Recent process guidelines and standards such as the Integrated Capability Maturity Model (CMMI) [5], ISO/IEC 12207 for software engineering [11] and ISO/IEC 15288 for systems engineering [12] emphasize the need to integrate systems and software engineering processes. Various curriculum guidelines, such as SWEBoK 2004 [18], GSwE 2009 [9], and SEBoK draft 2010 [16], show increasing integration between systems and software engineering. In addition, software engineering and systems engineering have many commonalities such as requirements development, design, integration, verification and validation, testing, and maintenance. Hence, software engineers need to play a stronger role in systems engineering. Thus, it is important to inject the concept of systems engineering into software engineering classes. Our two-semester USC core software engineering project course supports this concept by providing lectures, tutorials, assignments, project activities, tools, and feedback from course staff to equip the software engineering students with systems engineering knowledge and experience. Additionally, we use the Incremental Commitment Spiral Model, which is applicable to both systems and software engineering, as the process model.

This paper elaborates on how the systems engineering concepts are embedded and embraced in the software engineering class. Section 2 describes the nature of the software engineering class and the system and software process model. Section 3 reports the similarities between systems engineering concepts and the class materials. Section 4 describes how the course staff helps the students to come up the systems engineering learning curve. Section 5 shows feedback from the students and the clients. Section 6 presents the resulting conclusions.

2. Background and Related Work

2.1. The MSCS-SE Core Software Engineering Project Course

CSCI577ab [6] is the core software engineering project course within the University of Southern California (USC)’s Master of Computer Science with Specialization in Software Engineering (MSCS-SE). The main objective of the course is to prepare students for software leadership careers through the 2040’s. Software Engineering I or CSCI577a in the Fall semester focuses on software-intensive systems engineering, including system operational concept formulation, requirements negotiation and definition, prototyping, COTS and services evaluation and selection, system and software architecture definition, life cycle plans and processes, risk analysis, feasibility analysis, and verification and validation. Software Engineering II or CSCI577b in the Spring semester focuses on software product creation, integration, test, documentation, transition, and maintenance with an emphasis on quality software production.

Figure 1. Overview of Software Engineering Project Class Milestones. As shown in Figure 1, before the fall semester starts, the course staff reviews the project proposals submitted by potential clients. If the project will not fit into a 2-semester software engineering class, the proposal will be dropped. This is done by the first milestone, the Exploration Commitment Review (ECR). The second milestone is the Valuation Commitment Review (VCR), which will be conducted after the student teams analyze the clients’ current business processes and explore alternatives. The third milestone is the Foundations Commitment Review (FCR), which evaluates whether the project is feasible and ready to define various project foundations such as the system and software architecture and the project plans. The Development Commitment Review (DCR) is the fourth milestone which evaluates whether the team has built the right foundations, including the detailed architecture and plans for the Development phase. To accommodate the possible client and team changes over the semester break, the fifth milestone, Rebaselined Development Commitment Review (RDCR) is held to confirm that all the success critical stakeholders are committed to the foundations and that the team is ready to implement the product based on the agreed foundations. A Core Capability Drivethrough (CCD) is held after the first development iteration in order to check the usability and completeness of the core functionalities. The Transition Readiness Review (TRR) evaluates whether the team is ready to perform system and software transition. The Operations Commitment Review (OCR) evaluates whether the clients will be able to successfully operate and maintain the system and software.

2.2. The Incremental Commitment Spiral Model

The Incremental Commitment Spiral Model (ICSM) [4][14] is a refined version of the original spiral process model [1]. ICSM covers the full system development life cycle consisting of five incrementally-defined life cycle phases (Exploration, Valuation, Foundations, Development, and Operations phases). The ICSM, as shown in Figure 2, is not a single one-size-fits-all model but a risk-driven framework for tailoring a process that best fits a project’s situation by using the risk-based decision options at the end of each spiral. The ICSM integrally includes software development, and it is currently used as the process model for system development in our software engineering class.

Figure 2. Overview of the Incremental Commitment Spiral Model The four underlying principles of the ICSM are:

1) Stakeholder value-based system definition and evolution – The project should be developed based on satisfying the value propositions of all success-critical stakeholders. Otherwise, the stakeholders will frequently not commit to their project roles, which will lead to project rejection or major rework.

2) Incremental commitment and accountability – Stakeholders do not commit to a single pre-defined set of requirements and resource contributions, but commit incrementally as the nature of the system is better understood. Otherwise, the project often becomes locked into out-of-date concepts of what the system should provide its stakeholders, leading to project rejection or major rework. 3) Concurrent system and software definition and development – Contrary to

sequential development, the concurrent development of requirements, solutions, hardware, software, and human factors allows the project to move faster, avoid premature commitments, and be more flexible to yield the best results.

4) Evidence and risk-based decision making – Making the evidence of project feasibility a first-class deliverable provides a way to synchronize and stabilize the concurrently-defined system elements. Shortfalls in evidence are uncertainties that identify the level of risk of proceeding without stronger evidence of project feasibility.

Table 1. Required skills of a systems engineer Key skills of a systems engineer [9] Characteristics of a successful systems engineer [19]

Acquiring systems engineering skills through ICSM Practices

Identifying the success-critical stakeholders

Understanding the system from multiple perspectives

Students identify success critical

stakeholders such as end users, maintainer, and administrator by using CRACK model (Collaborative, Representative, Authorized, Committed, and Knowledgeable )

Determining key stakeholders’ value propositions

• Understanding the system from multiple perspectives

• Understanding a new system/concept during its presentation

WinWin negotiation is the key activity for brainstorming stakeholders’ win conditions and constraints. Added information is drawn from other client interaction activities such as team meetings, prototyping, business process identification, business case analyses, and acceptance test planning Helping stakeholders collaborate in negotiating a mutually satisfactory set of specifications Understanding the system from multiple perspectives

Win conditions are used as a starting point in defining a mutually satisfactory set. Moreover, risk analysis, architecture review board, prototypes, life cycle plan, feasibility evidence development are all key ingredients for software engineering teams to exercise this skill.

Adapting the plans and specification in mutually satisfactory ways to respond to changes Understanding the implications of proposed changes

Weekly project plan updates, change and configuration management, verification and validation, and peer review are examples of activities that allow student teams to plan and re-plan according to the project status Managed Problem

Solving • Understanding systems synergy

• Understanding limits to growth

This skill can be acquired through various project activities such as risk identification and management, quality management, configuration management, team

management, project planning and tracking Engineering

Problem Solving •

Understanding the systems without getting stuck on the details

• Understanding analogies and parallelism between systems

Student teams are required to apply their engineering knowledge to develop a product that will satisfy the clients’ requirements. Various activities allow student teams to exercise this skill such as operational concept engineering, requirements engineering, system and software architecting, process engineering, and test engineering

Strategic Problem

Solving •

Understanding the whole system and seeing the big picture

• Understanding interconnections; closed-loop thinking

• Thinking creatively

To tackle the problems, student teams need to set up a strategy to plan, implement, monitor, and solve problems during their project development. This skill can be achieved through win-win negotiation, analysis of alternatives, business case analysis, life cycle planning, and solution development

3. Systems Engineering Coverage in the Software Engineering Class

3.1. Systems Engineering Skills and Problem Solving Approach

To prepare students for the systems engineering tasks, students need to learn and acquire various skills through practice. Table 1 provides information about ICSM activities that relate to the required systems engineering skills based on GSwE 2009 [9] and characteristics of a successful systems engineer [19].

3.2. Systems Engineering Roles

By taking roles, students obtain first hand experience in learning and internalizing their systems engineering responsibilities and obligations. Table 2 compares twelve systems engineering roles that are suggested by Sheard [17] to the roles in our software engineering class. Please note that, the twelfth role, a classified ads systems engineer, refers to job description of a systems engineer posted in the classified ads, which could represent any tasks or unrelated tasks that are not in the first eleven roles. Table 2 shows that systems engineering roles and software engineering are fairly compatible. However, there are other roles that are not mentioned in [17] system engineering but appear in software engineering projects, such as COTS or Non-Developmental Item (NDI) evaluator and acquirer. On the other hand, with the limitation of the class schedule, students may play only a partial role in coordinating the transition to client logistics/operations, maintenance, and disposal engineering.

Table 2. Systems engineering roles

Twelve Systems Engineering Roles [16]

ICSM Roles

Requirements Owner Requirements Engineer / WinWin Negotiation Shaper System Designer System Architect / Prototyper / UML modeler

System Analyst Operational Concept Engineer/ Feasibility Analyst Validation/Verification Engineer Verification and Validation Personnel / Tester Logistics/Operational Engineer Life Cycle Planner/ Transition Coordinator

Glue Among Subsystems Builder

Customer Interface Operational Concept Engineer / Project Manager

Technical Manager Feasibility Analyst/ Life Cycle Planner / Project Manager Information Manager Quality Focal Point

Process Engineer Life Cycle Planner/ Feasibility Analyst

Coordinator Project Manager/ Life Cycle Planner

3.3. Systems Engineering Activities, Artifacts and supporting tools

As part of the system development, with various supporting tools, students have to produce various artifacts, which will allow them to have hands-on experience in developing core-essential industrial type artifacts. Table 3 shows a complete mapping between system engineering activities [8][9] and the project artifacts produced by the team and the supporting tools provided by the class.

Table 3. Project artifacts and tools supporting systems engineering activities

Systems Engineering Activities [8] [9]

ICSM Project Artifacts _{Supporting Tools} ICSM

Requirements

Analysis •_• Operational Concept Development _{System and Software Requirements} Description

• WinWin Negotiation Report • Prototype Report

• WikiWinwin Tool • Benefit Chain

Diagram

System

Architecture • System and Software Architecture Description

• UML diagrams

• IBM Rational

System Architect System

Implementation •_• Prototype Functioning system

• Iteration Plan

• Iteration Assessment Report

• iRise • Subversion • Development

team’s choice Integration &

Verification •_• NDI-Interoperability Report Agile Review Forms

• Verification and Validation Report

• Bugzilla • iStudio

Transition &

Validation •_• Agile Review Forms _{Transition Plan}

• Training Materials • Release Description

• Bugzilla

Use, Support,

Disposal •_• User Manual Support Plan

• Regression Test Package

• Development

team’s choice Project

management •_• Weekly Project Plan Weekly Progress Report

• Weekly Risk Identification and

Analysis

• Weekly Effort Report • Life Cycle Plan

• Feasibility Evidence Description • Support Information Document

• Microsoft Project • DART • Effort Reporting System • COCOMO II • COCOTS Overall

Coordination •_• Support Information Document Executive Summary

• ICSM Electronic

Class lectures, class activities and project activities also help students to prepare themselves for systems and software engineering roles. Table 4 shows that class and project activities provide full coverage on systems engineering activities. The first column represents system engineering life cycle activities. The second column represents the lectures, readings and assignments that are covered in the software engineering class. The last column represents activities that are conducted in the class.

Table 4. Systems engineering topics coverage in software engineering class

Systems Engineering Activities [9]

ICSM Topics Coverage ICSM

Project Activities Coverage

Requirements

Analysis •Operational Concept Development

•Stakeholder Benefits and

Business Case Analysis

•Requirement Negotiation and

Definition • WinWin Negotiation • Client/Team Meetings • Prototyping • Requirements Negotiation Tool Training System

Architecture •NDI and Legacy System Analysis and Tailoring

•Object Oriented Analysis and

Design

• NDI Evaluation

• Architecture Tool Training • System and Software

Architecture Workshop System

Implementation •Concurrent Increment Development and V&V; Next-increment Rebaselining

•Design Patterns

•Object Oriented Development

• Concurrent Engineering • Requirements-Design-Code

Peer Reviews, V&V

• Core Capabilities Drive

through Integration &

Verification •_•NDI-Interoperability Traceability Matrix

•Software Testing •Quality Management

• Requirements-Design-Code

Peer Reviews

• Integrated Independent

Verification and Validation (IIV&V)

Transition &

Validation •New-System Cutover, Installation, and Training

•Design-Code Review •User Acceptance Test •Quality Management

• Core Capabilities

Drivethrough

• Integrated Independent

Verification and Validation (IIV&V)

• Cutover and Training

Use, Support,

Disposal •_•System Transition and Support Evolutionary requirements

•Regression Test Package

• Core Capabilities

Drivethrough

• Transition Coordination

Project

Management •_•Cost Estimation Business Case Analysis

•Project Management, Team

Collaboration

•Project Contracting

• Cost Estimation Training • Project Planning Training • Mentoring sessions

4. Feedback from course staff

Feedback enables the students to learn about their status and to improve their performance. As project progresses, student teams will receive feedback from course staff and other stakeholders in many different ways.

• Architecture Review Board (ARB) – ARB is one of the best practices developed by

AT&T, and it is a widely used practice in systems engineering [14]. There are 2 ARBs which mark as a milestone review in each semester. At each ARB, student teams show the project feasibility evidence and project progress in the form of the agreed requirements, operational concepts, prototypes, design and architecture, project plan, business case analyses, test cases, test results, risks and mitigation plan, and strengths, weaknesses of the team . Student teams will present all their work in front of the architecture review board, which are course instructors, clients, teaching assistants, graders, and related stakeholders. The two ARBs in each semester can be compared to mid-term and final examinations. The ARB will check their evidence based on the exit criteria of each milestone and provide feedback to the team for future improvement.

• Mentorship – Mentors are PhD students who took software engineering class and perform

research in software engineering. Student teams can ask for additional explanation or suggestions on problems they are facing. With peer-like suggestions, students are eager to ask more questions, hence gain more knowledge and experience in their project development.

• Workshops – Besides lectures, there are special workshops on topics that can be learned

better in an interactive session. Students are encouraged to bring in their project artifacts, current problems and concerns. Teaching staff provides feedback or extra support.

• Focus group – Outside the class room, teaching assistants organize special sessions for

students to speed up the learning curve for certain roles such as system and software architect, shaper, or feasibility analyst. The discussion in the focus group includes case studies, previous project examples, and exception case discussion.

• Artifact grading – Teaching assistants and graders grade project artifacts and homeworks

with detailed explanations. Additionally, the teaching assistants and faculty provide feedback on their core capabilities drivethrough and design-code review results.

• Verification and Validation Personnel – Off-campus students are usually full-time

employees and generally have more experience in system and software development. Off-campus students take a project V&V role by evaluating the project artifacts and filing defects and concerns in the defect tracking system, and by virtually participating in ARB reviews.

• Discussion Board - An online discussion board allows students to post their questions and

concerns or discuss class-related topics, and get feedback from course staff, team members and classmates.

• ICSM EPG – The Incremental Commitment Spiral Model Electronic Process Guide[10] is

a major source of information for the software development guidelines. The ICSM EPG provides information about each role and responsibilities, tasks, work products, practices, checklist, and artifact templates. Student teams can check their progress or their work based on the guidelines and examples provided in the ICSM EPG.

5. Project Results and Feedback from Students and Clients

One way to learn whether the project is successful is to look at the feedback from the stakeholders. Two key stakeholders of the real-client project software engineering class are the students themselves and the clients. Based on the feedback from students in their end-of-semester reflection-in-action individual critiques, the students are largely very satisfied with

the knowledge and experiences they learned from the class, but also helpful in identifying future course improvements. With their role-playing responsibilities and real-life clients, they learn how to determine and handle stakeholders’ wants and needs from hardware, software and peopleware perspectives and translate them into a successfully operating system.

Another most important stakeholder is the clients, who closely participate in the whole development life cycle. They learn to give their win conditions, to compromise to gain mutually satisfactory sets of plans and specifications, and to discover new concepts and knowledge about the system. At the end of semester, we ask the clients about their satisfaction on the team performance in various activities. The results, in Table 5, have shown that the clients are overall very satisfied with the team in all system engineering perspectives. Trends are generally upward, but scores vary from year to year based on course content evolution, client satisfaction thresholds, and student preparedness levels.

Table 5. Clients' evaluation (2006 – 2009)

Questions 2006 2007 2008 2009 Average

(Scale of 5) 1. Operational concepts: How well did the team capture

your ideas of what the new system should do? 4.71 4.83 4.56 4.69 4.70

2. Stakeholders' value propositions: Did the team suggest new idea about your project? Did the team support

you in any project complexity? 4.29 4.45 4.59 4.23 4.39

3. Team responsiveness: Was the team responsive to you and to your requirements? Did the team answer any questions you might have had? How successful was the

requirements negotiation between you and your team? 4.56 4.92 4.50 4.69 4.67

4. Project satisfactory: How satisfied are you with the prototype? How well does the proposed system meet the

need identified by your project? 4.29 4.36 4.19 4.38 4.30

5. Project feasibility: Do you think this project should be carried forward for development in CS577b? Has the team adequately identified and managed the potential risks and complications associated with the project? Do you foresee difficulties in transitioning this project into the

development and implementation phase? 4.59 5.00 4.50 4.62 4.68

6. Team communication: How effective was the team in communicating with you? Do you feel you had enough meetings with them? Did the team make effective use of your time and expertise? What means did you use to

communicate? 4.11 4.54 4.56 4.70 4.48

6. Conclusions

A successful software engineer should know not only how to engineer software, but also how to participate in engineering the other system elements. The increasing importance of, and rapid changes in information and software technology make systems-oriented software engineers critical to the success of most future system developments. To design a course for them, it is very important to supply the knowledge, improve their skills, and provide experiences for them to be ready to participate in industry projects on day one.

Section 3.1 indicates that students can learn the key skills required for a systems engineer from assorted class activities. Section 3.2 shows the compatible mapping between systems engineering roles and software engineering course roles. Section 3.3 presents the match of

key systems engineering activities to project artifacts and tools usage in the software engineering class. Section 3.4 provides evidence that knowledge about systems engineering activities is considerably covered by lectures, presentations, and activities in the software engineering class. Section 4 explains how feedback from teaching staff helps students to acquire system engineering and development skills. Section 5 reports the satisfaction feedback about systems engineering elements from the students and the clients.

Generally, our USC software engineering course has integrated the concept of systems engineering, both in the lectures, and in student teams’ project development activities, artifacts, and supporting tools. However, with the limitation of the class schedule that the semester ends slightly after the students deliver their final product, students usually do not have a chance to fully experience system operation, maintenance and disposal, which are also key activities in systems engineering – although clients often hire students to help them evolve the systems that they have developed.

6. References

[1] Boehm, B., "A Spiral Model of Software Development and Enhancement," IEEE Computer, Volume 21, Issue 5, May 1988, pp. 61-72

[2] Boehm, B., "Unifying Software Engineering and System Engineering," Computer, Volume 3, Number 3, March 2000, pp. 114-116

[3] Boehm, B., "The Future of Software and Systems Engineering Processes," SSCI Member Forum, 2005 [4] Boehm, B. and J.A. Lane, “Using the Incremental Commitment Model to Integrate Systems Acquisition,

Systems Engineering, and Software Engineering,” “Cross Talk, October 2007, pp.4-9 [5] Chrissis, M., B., Konrad, M., Shrum, S.: CMMI. Addison Wesley (2003)

[6] CSCI577ab Software Engineering Class Website,

http://greenbay.usc.edu/csci577/fall2008/site/index.html

[7] Dahmann, J., G. Rebovich Jr., J.A. Lane, "Systems Engineering for Capabilities," CrossTalk Journal, Volume 21, Number 11, pp. 4-9

[8] Dahmann, J., G. Rebovich Jr., J.A. Lane, "System Engineering Artifacts for SoS," IEEE Systems 2010 Conference

[9] GSwE 2009 curriculum guidelines

http://www.gswe2009.org/fileadmin/files/GSwE2009_Curriculum_Docs/GSwE2009_version_1.0.pdf [10]Instructional ICSM-Software Electronic Process Guide, http://greenbay.usc.edu/IICSMSw/index.htm [11]ISO (International Standards Organization).: Standard for Information Technology – Software Life

Cycle Processes. ISO/IEC 12207 (1995)

[12]ISO (International Standards Organization).: Systems Engineering – System Life Cycle Processes. ISO/IEC 15288 (2002)

[13]Klappholtz, D., J. McDonald, A. Pyster, "The Graduate Software Engineering Reference Curriculum (GSwERC)," cseet, pp.290-291, 2009 22nd Conference on Software Engineering Education and Training, 2009

[14]Maranzano, J., et.al, "Architecture Reviews: Practice and Experience," IEEE Software, vol. 22, no. 2, pp. 34-43, Mar./Apr. 2005

[15]Pew, R. W. and A.S. Mavor, “Human-System Integration in the System Development Process: A New Look”. 2007, National Academy Press.


[16]SEBoK
2010, Systems Engineering Body of Knowledge – Version 0.25 - http://www.bkcase.org/

[17]Sheard, Sarah A. “Twelve Systems Engineering Roles,” Proceedings of INCOSE, 1996.

[18]SWEBoK 2004 Guide to the Software Engineering Body of Knowledge – http://www.swebok.org

[19]Walden,
D.,
“Systems
Engineering:
Fundamental
Skills
and
Why
They
are
Important,”


Seventeenth
Annual
Professional
Development
Seminar,
(St.
Paul,
MN,
USA,
March,
2010),
The
 Society
of
Women
Engineers
Minnesota
(SWE‐MN),
2010.