Software Acquisition Management Guidelines

S

OFTWARE

A

CQUISITION

M

ANAGEMENT

G

UIDELINES

by

Daniel Svennberg

A Thesis

submitted for the Degree of

Master of Science

Linköping, Sweden, 2001

The Department of Computer and Information Science

Linköping University

LiTH-IDA-Ex-01/32

Advisors, Q-Labs AB

Kristian Sandahl

Copyright 2001 Daniel Svennberg

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means including, without limitation, photocopying, recording, or oth-erwise without the prior written permission of the author. Written

per-Linköpings Universitet,

Institutionen för Datavetenskap

Q-Labs, Linköping and Maryland

Fraunhofer, Maryland,

Center for Experimental Software Engineering

LiTH-IDA-Ex-01/32

Linköping University

Q-Labs AB

i

Abstract

The thesis provides guidelines from the viewpoint of the cus-tomer for managing an acquisition project concerning an out-sourced development of a software product customized for the specific needs of the customer. Current frameworks for soft-ware acquisition management such as SA-CMM, IEEE 1062, ISO 12207, etc. as well as literature, articles, and interviews has been used as a basis for formulating the guidelines.

The resulting guidelines include acquisition team roles descrip-tions, checklists for artifacts, customizable processes, and cus-tomization factors. The processes contain guidelines for the fol-lowing issues: project steering, project management, planning and organizing, acquisition training, requirements ment, risk management, contractor selection, contract manage-ment, product evaluation, transition to support, and post par-tum.

The guidelines can be used as a basis for planning an acquisi-tion project or be read as an overview over the many issues that concern software acquisitions.

Many thanks to (in alphabetical order)…

Fraunhofer Institute, Jaak Urmi, John Marciniak, Kristian San-dahl, Lars Ljungberg, Linköping University, Mats Ran, Mikael Lindvall, Q-Labs, Victor R. Basili, Winifred Menezes, Östen Os-karsson, and everybody else

… for your help making this thesis possible.

iii

Table of Contents

Abstract...i

Acknowledgements ...ii

1

Introduction...1

2

Software Acquisition Environment...6

2.1 The nature of software ...6

2.2 Software acquisition...7 2.3 Software development ...8 2.4 Outsourcing...9 2.5 Common problems ...13 2.6 Frameworks ...16

3

Customization ...20

4

Roles ...29

4.3.7 Maintenance and support staff ...34

4.3.8 Independent verification and validation...35

4.3.9 System engineering and technical assistance ...35

4.4 Executing roles...36 4.4.1 Contractor ...36 4.4.2 Collaborating contractor...37 4.4.3 Subcontractor...37 4.4.4 Supplier ...37

5

Artifacts ...38

5.2.1 Request for proposal...47

5.2.2 Contractor evaluation criteria ...49

5.2.3 Proposal...50

Table of Contents

© Daniel Svennberg 2001 v

5.3.1 Artifacts from the contractor ...53

5.3.2 Product evaluation plan...54

5.3.3 Product evaluation report...55

5.3.4 Change request...55

5.3.5 Project status report...56

5.4 Finalization...57

5.4.1 Deliverables...57

5.4.2 Post partum plan...57

5.4.3 Post partum...58

6

Processes...60

6.1 Overview...60

6.2 Project steering ...66

6.3 Project management ...69

6.4 Planning and organizing ...72

6.5 Acquisition training ...74 6.6 Requirements management ...76 6.7 Risk management...78 6.7.1 Risk identification ...80 6.8 Contractor selection ...85 6.8.1 Contract types ...88 6.9 Contract management...96

6.9.1 Project status metrics...100

6.9.2 Earned value...102

6.10 Product evaluation ...106

6.10.1 Product evaluation approaches ...109

6.11 Transition to support...110

6.12 Post partum...113

7

Results and summary ...116

7.1 Guidelines summary ...116 7.2 Outlook ...122

8

Bibliography...124

A

Frameworks coverage...129

A.5 ISO 12207 ...133

A.6 ISO 15504 ...134

A.7 Euromethod ...135

1

1 Introduction

“Much of present-day software acquisition procedures rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it. I think this assumption is fundamentally wrong, and that many software acquisition problems spring from that fallacy.”

– Frederick P. Brooks, Jr.

This chapter states the background of the thesis, its objective, the approach, and outlines the contents of the report.

1.1 Background

Many organizations have a need to acquire customized soft-ware products that meet special demands that no commercial off-the-shelf product can meet. Many times the most efficient and economically viable solution to obtain the software is to outsource the development to a contractor instead of develop-ing the software in-house.

In order to manage such a software acquisition project success-fully, many different non-trivial issues such as requirements elicitation and management, risk analysis and management, contractor selection, contract management, product evaluation, etc. need to be addressed. Several frameworks and standards have been developed for assisting and assessing such software acquisition efforts.

The purpose of this thesis is to gather practices from these frameworks and other sources and to present them as guide-lines for software acquisition project managers. The thesis along with the diploma thesis (Assmann, 2000) of a German student, Danilo Assmann, are parts of an international project on soft-ware acquisition between Linköping University in Sweden, Q-Labs in Sweden and USA, and the Fraunhofer Institute in Germany and USA. The thesis by Assmann deals mainly with the evaluation and selection of a contractor, while this thesis deals mainly with the management of an acquisition project.

It isn’t possible for a single master’s thesis to cover this subject in all its intricacies, therefore the guidelines should be consid-ered as an introduction to software acquisition management, and hopefully as an inspiration on further studies on how to manage software acquisition projects.

1.2 Target

audience

The primary audience group for this master’s thesis consists of those who are assigned the task of managing a software acqui-sition project. Naturally, all groups that are involved in soft-ware acquisition projects may find this thesis interesting.

1.3 Objective

The objective of this master’s thesis is to provide guidelines from the viewpoint of the customer for managing an acquisition project concerning an outsourced development of a software product customized for the specific needs of the customer.

1.4 Problems

The objective stated above can be broken down into seeking an-swers to the following problems:

1 Introduction

© Daniel Svennberg 2001 3

• What activities should be performed? • What roles are conceivable?

• What artifacts are conceivable?

• How can the activities, roles, and artifacts be customized for different kinds of projects?

1.5 Scope

The management of custom-made software product acquisi-tions will be examined – not services. Fully or partially devel-oped products are the main concern – not commercial off-the-shelf products. The acquirer’s point of view is mainly ad-dressed – not the contractor’s. Legal aspects will not be exam-ined further than contract contents. The issues will be examexam-ined from a business perspective instead of a governmental or mili-tary perspective.

1.6 Approach

An introductory study of the subject was performed by reading the thesis of the German student (Assmann, 2000), the different frameworks listed in section 2.6, Frameworks, on page 16, web pages, and articles on software acquisition, and performing in-terviews with Jaak Urmi, Östen Oskarsson, and Mats Ran. This gave an understanding of the concepts of software acquisition management and a number of ideas on how to formulate the guidelines and provide answers to the problems stated above. I found that the different sources gave quite a coherent picture of software acquisition management without many contradic-tions. The sources contained mainly different descriptions of the same issues. Like Assmann (2000) does in his thesis, I as-sume that the frameworks are useful in their own context and therefore are viable as input to the guidelines. Furthermore my goal was to extract and assemble the lion’s share of the advice

given in the main frameworks SW-CMM, SA-CMM, FAA-iCMM, IEEE 1062, ISO 9000, ISO 12207, ISO 15504, and Eu-romethod, with as little filtering as possible, and then augment with advice from the other sources studied. Naturally, I had to integrate and arrange the advice into a suitable and coherent form for the purpose of the guidelines.

The ideas on how to describe the guidelines were presented and discussed with several people, including Kristian Sandahl, Lars Ljungberg, Danilo Assmann, John Marciniak, Jon Valett, Winifred Menezes, Mikael Lindvall, and others. These discus-sions motivated me to write the guidelines in their present form.

The guidelines described in this thesis are not a new model for software acquisition management, but an attempt to extract, as-semble, and integrate advice from the existing models and frameworks and present them as management processes, check-lists for artifacts, descriptions of roles, and customization fac-tors. The option would have been to describe the guidelines in a more prosaic fashion, but that would have made the sugges-tions for customization less logical and to the point.

1.7 Outline

The following is a description of the contents of each chapter in the report:

Chapter 1, Introduction, on page 1 describes the background of

the thesis, its objective, and approach.

Chapter 2, Software Acquisition Environment, on page 6

in-vestigates the nature of software, different types of software ac-quisitions, software development, outsourcing issues, problems with software projects, and software acquisition frameworks.

Chapter 3, Customization, on page 20 describes a number of

formal-1 Introduction

© Daniel Svennberg 2001 5

Chapter 4, Roles, on page 29 describes the different roles that

are conceivable for a software acquisition project.

Chapter 5, Artifacts, on page 38 contains checklists for the

dif-ferent types of artifacts that are conceivable for a software ac-quisition project.

Chapter 6, Processes, on page 60 contains suggestions on

cus-tomized software acquisition management processes.

Chapter 7, Results and summary, on page 116 summarizes the

2 Software

Acquisition

Environment

“A typical software project can present more opportunities to learn from mistakes than some people get in a lifetime.”

– Steve McConnell

This chapter introduces the concept of software acquisition, its nature, environment, and problems. It also lists some frame-works that deal with software acquisition management.

2.1 The nature of software

There are a number of inherent characteristics of software that affect the acquisition and development of customized software in a profound way. The most salient of these characteristics, in my opinion, are the following:

™Software is an almost infinitely malleable conceptual con-struct. As a consequence this makes it difficult to settle the requirements and makes the scope of the software prone to change.

™Software has no physical features and is therefore difficult to visualize. Brooks (1995) tells us that this makes it hard to design the software and to communicate the design of the software, which in turn makes it difficult to estimate

2 Software Acquisition Environment

© Daniel Svennberg 2001 7

makes it difficult to see progress in the development of the software.

™Software is extremely cheap to manufacture, as Reeves (1992) argues. All it takes is to transform the source code into executable code and clone as many copies as you wish. The actual costs for software lie in development and maintenance.

™Software is extremely complex. This is, according to Brooks (1995), due to the fact that software has a very large number of possible states, which increases exponen-tially with size. This makes it very hard to get an over-view of the software, which in turn makes it hard to de-sign the software to fully meet all quality requirements and to have full control over the possible errors.

2.2 Software acquisition

There are several different ways to acquire software. According to IEEE 1062 (1998), software products can be classified accord-ing to the degree to which the acquirer can specify the features of the software.

Commercial-off-the-shelf software, or COTS, is commercially

available software. It is usually well defined and documented and a wide range of commercial users has demonstrated its fit-ness for use. The supplier isn’t normally willing to modify the software for the needs of a specific customer, and also controls the maintenance of the software. The cost to acquire the soft-ware is low to medium and the delivery time is immediate. The customer has no control of the quality characteristics of the software.

Modified-off-the-shelf software, or MOTS, is similar to COTS

software with the difference that the supplier is willing to make modifications to the software product according to the cus-tomer’s requirements. The software product’s fitness for use has been demonstrated in similar applications. The customer

has some control of the maintenance of the product and its quality characteristics, i.e. the customized part. Delivery time is short to long and the cost to acquire the software is medium to high.

Fully developed software are unique, one-of-a-kind or

low-volume applications fully customized for the specific needs of the customer. The software product’s fitness for use is unprece-dented, but the customer has full control over its maintenance and quality characteristics. The cost to acquire fully developed software is high and the delivery time is long.

The characteristics of these product classes are summarized in the table below:

Table 1. Software acquisition types. FD stands for fully developed software.

This thesis applies to MOTS and especially fully developed software products.

2.3 Software development

To be brief, software development essentially consists of four activities: analysis, design, coding, and testing. There are a

number of methods that describe how and when these activities should be performed, ranging from code and fix, which is the

COTS MOTS FD*

Scope Fix _customizablePartially _customizableFully

Fitness-for-use Demonstrated _{similar applications}Demonstrated in Unprecedented

Maintenance No control Some control Full control

Delivery time Immediate Short-Long Long Acquisition cost

(Not usage/maint. cost) Low-Medium Medium-High High Quality (ISO 9126) Not controllable Partially controllable Mostly controllable

2 Software Acquisition Environment

© Daniel Svennberg 2001 9

most well known method, the waterfall method, which in its purest form is to perform the activities in the following sequen-tial manner: analyze, design, code, test, and never go back to a previous step.

More recently, iterative, evolutionary, or incremental develop-ment methods such as eXtreme Programming (Beck, 1999) have become popular. Following such a method means that the pro-gram is developed in a series of iterations or increments with a subset of the requirements implemented in each iteration; this has the benefit that you can more easily change your mind about the requirements as the program evolves. See the figure below for a comparison between the waterfall model, the spiral model and extreme programming.

Different software development methods suit different types of projects, but that is not for this report to examine. What is im-portant is that the contractor can produce good results with the method they use.

Figure 1. The development cycles of the waterfall model, the spiral model and extreme programming (Beck, 1999).

2.4 Outsourcing

There are many different reasons for outsourcing a software development project, such as the following examples:

Analysis Design Coding Testing Waterfall Spiral XP Scope

• It’s difficult to hire developers. • It’s cheaper than hiring developers.

• Somebody else can develop the program faster, cheaper, and with higher quality than you can.

• You lack the necessary competence and resources. • You want to use your staff for other tasks.

The following are examples of drawbacks with outsourcing: • You lose some control over the software development

process, which implies that the risk is higher; possibly you risk putting the fate of your competitiveness in the hands of the contractor.

• You don’t build up your own software development competence.

• You don’t get any indirect labor hours since you don’t have your own development staff.

Haimes, et al. (1997) states that there are three groups of partici-pants in a software acquisition project, each with separate goals:

• The end user wishes to have a product that function well in operation. Another possible goal for the end user is to get the product as quickly as possible.

• The customer, that is the participant that finances the pro-ject, can have several goals: Similarly to the user, the cus-tomer wishes to receive a software product that function well in operation. The customer also wants to minimize cost and deviation from time schedule. Furthermore, the customer wants to maximize return on investment and to have a well-functioning relationship with the contractor.

2 Software Acquisition Environment

© Daniel Svennberg 2001 11

• The contractor on the other hand often wants to maximize profits and to maximize the potential for future earnings, such as additional contracts.

This gives us the optimization problem that a software acquisi-tion project consists of, with several of the goals overlapping and conflicting. For instance, the customer wants to maximize the technical performance of the product and on the same time minimize cost and deviation from time schedule. Furthermore, the cost minimization goal of the customer conflicts with the profits maximization goal of the contractor. Obviously trade-offs are necessary to have an operational contract.

Participant Objective

End user Maximize technical performance Minimize development time Customer Maximize technical performance

Maximize customer-contractor relation Maximize return on investment

Minimize cost

Minimize deviation from time schedule Contractor Maximize profit

Maximize future earnings

Table 2. Participants in a software acquisition project and their goals. Adapted from Haimes, et al. (1997).

The most common scenario when you think of an outsourcing project is that you have two parties involved – a single cus-tomer and a single contractor. Gallivan (1999) shows that this simple dyadic relationship is not always the case, as more com-plex outsourcing constellations are becoming more common-place. Gallivan (1999) gives a taxonomy of outsourcing

constellations:

• The simple dyadic constellation consists of a single cus-tomer outsourcing to a single vendor.

• The multi-vendor constellation consists of a group of ven-dors collaborating in a project outsourced by a single cus-tomer.

• The co-sourcing constellation appears when a group of customers that have similar needs cooperate and jointly outsources a project to a single vendor.

• The complex constellation is a combination of the multi-vendor and co-sourcing constellations, where you have a group of customers jointly outsourcing to a group of col-laborating vendors.

Figure 2. A taxonomy of outsourcing relationships. (Gallivan, 1999)

To make the outsourcing scenario even more complex you can have one or more subcontractors involved and several support-ing contractors and suppliers.

Compared to the dyadic constellation, the multi-vendor rela-tionship allows vendor specialization and reduces the risk of a vendor behaving opportunistically since the other vendors are ready and willing to substitute a non-performing vendor.

C V C V _C V V V C V V V V V V C C C V C C C C C C V V V V C C C V V V V V V C C C C C C

One Vendor

Many Vendors

One

Client

Simple Dyadic

Multi-Vendors

Many

Clients

Co-Sourcing

Complex

One Vendor

One Vendor

Many Vendors

Many Vendors

One

Client

One

Client

Simple Dyadic

Simple Dyadic

Multi-Vendors

Multi-Vendors

Many

Clients

Many

Clients

Co-Sourcing

2 Software Acquisition Environment

© Daniel Svennberg 2001 13

Drawbacks are increased coordination costs and more complex contracts, etc. (Gallivan, 1999)

Compared to the dyadic constellation, the co-sourcing relation-ship allows risk sharing and reduction, and cost and time sav-ings due to buyer economies of scale. Drawbacks are increased coordination costs and that it is more difficult to control

information and keep secrets because of the interdependent nature of the relationship between the customers, etc. (Gallivan, 1999)

2.5 Common

problems

Marciniak and Reifer (1990) states that the main problems in software acquisitions are:

• Development costs exceeds budget. • Time schedule is overrun.

• Outcome does not meet expectations.

The Standish Group (1995) “Chaos” report investigated how common these problems are. 8 380 software development pro-jects were surveyed with respondents representing small, me-dium, and large companies across major industry segments such as banking, health care, and federal organizations, etc. The report does not indicate the level of outsourcing in these soft-ware development projects, but the statistics will still give an idea of the extent to which software development projects suf-fer from these problems.

The findings were:

• 16% of the projects were completed on time and on budget, with all features and functions as initially speci-fied. These projects were labeled as successful.

• 53% of the projects were completed and operational but over budget, over the time estimate, and offered fewer

features and functions than originally specified. These projects were labeled as challenged.

• 31% of the projects were canceled at some point during the development cycle. These projects were labeled as

canceled.

Figure 3. Findings from the Standish Group (1995).

In addition, the average project exceeded planned budget with 90% and schedule with 120%.

The root cause to these problems is primarily ineffective man-agement as stated in Brown, et al. (1998):

In 1987, the Defense Science Board concluded that these soft-ware acquisition problems came not from technical difficulties, but from poor management. Since 1987, the situation has gotten worse, not better. Software has gotten bigger, more complex, and more expensive, and ineffective management is still the root cause of much that afflicts software acquisition.

Below is a list of some common problems in software acquisi-tion projects. It is assembled and adapted from The Standish Group (1995), Ganssle (1998), Marciniak and Reifer (1990), Reifer (1997), and interviews with Urmi (2000), Oskarsson (2000), and Ran (2000):

• Laissez-faire – The customer does not take an active part in the project once the contract is signed.

Successful 16% Challenged 53% Canceled 31%

2 Software Acquisition Environment

© Daniel Svennberg 2001 15

• Administrative overload – Too much administratrivia to monitor contract compliance instead of focusing on tech-nical matters.

• Creeping scope – The customer keeps adding and chang-ing scope and functionality to a job in progress on a tight schedule with limited resources.

• Fragmentation – Members of the team, both customer and contractor, are pulled off to work on other projects at random times.

• Goldplating – Having overkill requirements or making sophisticated and new rather than simple and proven technical solutions.

• “I’m paid to engineer!” – The customer tells the contrac-tor how to do the job, not what the job is.

• Missing or bad indicators – Measurement of progress and overall performance is qualitative instead of quantita-tive. The performance levels of the indicators are poorly set.

• Who’s in charge? – Too many bosses, decisions aren’t made in a timely manner.

• No end user involvement – Not getting and incorporat-ing the end user’s point of view on the functionality and usability of the product.

• Poorly defined requirements – The customer gives an in-complete and unvalidated set of requirements to the con-tractor.

• Acquisition incompetence – Failure to understand the particular needs of software acquisitions.

• Overpromising – The marketing people of the contractor makes promises that the technical staff can’t keep.

• Lack of discipline – Reverting to an ad hoc development process when deadlines are getting close. “Gotta crank out that code!”

• Unrealistic expectations – Having an impossible sched-ule, and/or being unaware of the limitations of the tech-nology being used.

• Inadequate resources – Not having appropriate financial funding or lacking appropriate staff, tools, and equip-ment.

• No executive support – No support or interest in the pro-ject from top management.

• Lack of clear objectives – No clear statement of goal or vision causing project members to pull in different direc-tions.

• Ineffective communications – Lack of effective commu-nication channels, information not reaching the right peo-ple in a timely manner.

• Lack of competence – Not having the appropriate techni-cal and leadership skills.

• Friction – Cooperation is hindered by some of the in-volved parties not getting along for some reason.

All of these problems can probably be avoided by better soft-ware acquisition management practices. Also see the risk tax-onomy given on page 80.

2.6 Frameworks

The following is a list of frameworks that in some way gives guidance on software acquisition management. Apart from standards, books and collections of guidelines are mentioned.

2 Software Acquisition Environment

© Daniel Svennberg 2001 17

There are of course more frameworks that cover the subject than those listed1_.

SW-CMM – The Capability Maturity Model for Software is one of

the most well known frameworks for assessing the capabilities of software contractors. The purpose of the model is to help or-ganizations determine the maturity of their current software development capabilities. The model characterizes the level of an organization’s maturity based on the extent to which meas-urable and repeatable software engineering and management practices are institutionalized. The software acquisition activi-ties examined for this thesis are in the Software subcontract

man-agement key process area of the model. See Paulk, et al. (1993).

SA-CMM – The Software Acquisition Capability Maturity Model is

a framework developed to help organizations assess and im-prove their acquisition capabilities for software-intensive sys-tems. The structure of the model is consistent with SW-CMM in that it is a staged model with key process areas grouped in five maturity levels: initial, repeatable, defined, managed, and op-timizing. See Cooper, et al. (1999).

FAA-iCMM – The Federal Aviation Administration Integrated

Capability Maturity Model is an effort to combine the features of

SW-CMM, SA-CMM, and SE-CMM2 _{into one consistent model.}

It is intended to be used as a tool for organizations to evaluate their acquisition, management, and engineering practices and to devise improvements on them. See FAA-iCMM (1997).

CMMI – The Capability Maturity Model Integration framework is

an effort to integrate existing maturity models. CMMI is under development and will eventually consist of a series of disciplines ranging from software engineering and systems engineering to

1 _{www.software.org/quagmire is a good starting point to find out more} about available standards.

software acquisition, etc. A requirement for the CMMI frame-work is consistency with ISO 15504. This frameframe-work had not been extended to contain the software acquisition discipline when this thesis was written. See CMMI (2000).

IEEE 1062 – The IEEE Recommended Practice for Software

Acquisi-tion is a standard that takes a comprehensive approach to

giv-ing recommended practice for managgiv-ing software acquisition projects regarding COTS, MOTS, and fully developed software. It covers the whole acquisition process from planning organiza-tional strategy to using the software and is compliant with ISO 12207. See IEEE 1062 (1998).

ISO 9000 is a series of standards for quality management and

quality assurance. It is mainly intended to provide guidance in a situation where a contract between two parties require the demonstration of a contractor’s capability to develop, supply and maintain products. In this series, the standards that are im-portant for software development projects are ISO 9001 and ISO 9000-3. ISO 9001 – Quality systems – Model for quality assurance in

design / development, production, installation and servicing deals

with generic two-party contractual situations and ISO 9000-3

Guidelines for the application of ISO 9001 to the development, supply and maintenance of software gives guidelines for the application

of ISO 9001 to software projects. ISO 9001 is roughly compara-ble to SW-CMM level 2. See SS-ISO 9000-3 (1992).

ISO 12207 – The Standard for information technology – Software life

cycle processes takes a holistic and comprehensive approach to

the whole life cycle of a software product. It includes a well-defined terminology for software life cycles that is intended to standardize the terminology used by the industry. The standard can be used to establish software acquisition, supply, develop-ment, operation, maintenance, and supporting processes. It is also intended to foster an improved understanding between customers and vendors and among the parties involved in the life cycle of a software product. Worth mentioning is the goal of facilitating world trade in software. It replaces older U.S. mili-tary standards such as DOD-STD-2167A and DOD-STD-2168.

2 Software Acquisition Environment

© Daniel Svennberg 2001 19

ISO 15504 – The Information Technology – Software Process

As-sessment standard is a framework for assessing software

proc-esses regarding acquisition, development, support, etc. It is in-fluenced by and similar to SW-CMM in such that it has a capability scale. It is different from the SW-CMM in that it has not a discrete pass/fail mechanism and that it is aimed at assessing individual processes and not organizations. It is strongly aligned with ISO 12207. See ISO/IEC 15504 (1998).

Euromethod is a framework with the intent to assist an

acquisi-tion at the contractual level. It aims to help the customer and contractor understand each other, improve risk management, and harmonize the terminology used in an acquisition effort. According to Breu (1995), Euromethod is particularly aimed at guiding public procurement processes in the context of the open European market, but can be used for all types of software acquisition projects. See Euromethod (1996).

Other frameworks that provide important and useful advice for software acquisition management are the following: The Road to

Successful ITS Software Acquisition (Salwin, 1998), Software Acqui-sition Management (Marciniak and Reifer, 1990), The Program Manager’s Guide to Software Acquisition Best Practices (Brown, et al., 1998), Guidelines for Successful Acquisition and Management of Software-Intensive Systems (Mosemann, 1996), A Guide to the Pro-ject Management Body of Knowledge (Duncan, 1996), and Defense Acquisition Deskbook (DAD, 2000).

3 Customization

“Depending on the circumstances, you should be as hard as diamond, flexible as willow, smooth-flowing like water or as empty as space.”

– Morihei Ueshiba

This chapter investigates project categorization factors that in-dicate what level of formality is necessary for the actual soft-ware acquisition project.

3.1 Customization

levels

Software acquisition projects have different characteristics. Some projects are small, short-term, and easy to grasp, while others are the other way around. Cockburn (n.d.) states that

… each organization must develop its own way of working, and each project must get its own, tailored methodology, deal-ing with culture, location, criticality, and technology.

To have effective acquisition processes that give good results, it is beneficial to employ an adequate level of formality for those processes according to the characteristics of the actual project. For the purpose of this thesis I give suggestions on three levels of formality for the acquisition management processes. These levels are:

3 Customization

© Daniel Svennberg 2001 21

Minimal processes – The project is easily managed and the

purpose of the acquisition processes is achieved with a low level of rigor. The processes contain just the activities without which the project would really be in trouble. This level is suit-able for low-risk projects.

Controlled processes – The project has a tangible complexity

and a reasonable risk of going out of hands. To contain the complexity and risks, a higher level of formality is appropriate for the acquisition processes. The processes on this level contain the activities of the minimal processes and add some activities that are not absolutely necessary but make things more con-trolled and reliable. This level is suitable for medium-risk pro-jects.

Robust processes – The project is highly complex and has a

strong tendency to get chaotic unless it is managed appropri-ately and performed with discipline. The processes contain ac-tivities that increase the formality and administrative effort but in return make things more robust and reliable. This level is suitable for high-risk projects.

An acquisition process level below minimal would imply ad hoc, laissez-faire processes, which is not recommended.

3.2 Customization

factors

To determine what level of formality is appropriate for your project you can look at the customization factors in the follow-ing sections. The factors have been inspired by D.I.R. (2000), ISO 12207 (1995), and Euromethod (1996) and should be viewed as guidelines and not exact values on what level to choose. It is important to look at several factors and determine which are most important for your project before you decide upon a for-mality level.

3.2.1 Product characteristics

The product characteristics factors obviously stem from the char-acteristics of the software product to be acquired.

Requirements volatility

Since software development is a creative process, and it is diffi-cult to anticipate all technical problems beforehand, change of requirements occur more or less in all software development projects. However, due to the nature of the problem the soft-ware is to solve, the total set of requirements can be more or less change-prone.

Minimal proc-esses suffi-cient

Stable requirements. Highly unlikely that the overwhelming majority of the requirements will change.

Controlled processes recommended

A significant subset of the requirements is vola-tile.

Robust proc-esses rec-ommended

Volatile requirements. High degree of uncer-tainty on a large or important subset of the requirements.

Program size

The estimated program size indicates the effort that is needed to build the software. The values given in the table below are to be considered as suggestions since program size depends on what language is used and other factors.

Minimal proc-esses suffi-cient

Small. Less than 32 KLOC1_.

3 Customization © Daniel Svennberg 2001 23 Controlled processes re-commended Medium. 32 – 128 KLOC. Robust proc-esses rec-ommended

Large. More than 128 KLOC.

Criticality

Criticality is the impact malfunctioning software will have on people’s safety and the magnitude of financial loss that can re-sult, according to Oskarsson and Glass (1996).

Minimal proc-esses suffi-cient

People’s safety is not at risk, nor will any signifi-cant amount of money be lost if the software mal-functions.

Controlled processes recommended

People’s safety is not at risk, but there is a risk of moderate financial losses due to software mal-functioning.

Robust proc-esses rec-ommended

The safety of people is at stake or huge amounts of money. A disaster occurs if the software

doesn’t function properly.

Innovation

Oskarsson and Glass (1996) define innovative projects as those in which the problem is new and different and hasn’t been solved with computer programs before. Less innovative pro-jects use mature and proven technology.

Minimal proc-esses suffi-cient

Mature technology. Many similar applications exist.

Controlled processes recommended

The technology has been proven to work and is expanding but few similar applications exist.

Robust proc-esses rec-ommended

Emerging unproven technology where the ques-tion is more on whether the problem can be solved. Unique application.

Transformational impact

Transformational impact is the depth of change that the deliv-ered software product will have on the user’s behavior and the user organization’s processes and products.

Minimal proc-esses suffi-cient

Minimal change for the user.

Controlled processes recommended

Moderate modifications or amendments to the user’s behavior or the internal work processes and products of the user organization.

Robust proc-esses rec-ommended

Fundamentally changes the behavior of the user and the work processes and products of the user organization.

3.2.2 Effort

The effort customization factors delivery time, total cost, and

de-velopment team size are implied by the estimated effort to build

the software. The estimate is based on the product characteris-tics. The larger the effort is the higher formality is required to avoid unnecessary chaos.

Delivery time

Delivery time is the estimated calendar time for delivery of the finished software product. With a longer project there is a higher likelihood that new people will join the development team and others leave. Change in the requirements is also more likely. The values given in the table below are to be considered as suggestions.

3 Customization © Daniel Svennberg 2001 25 Minimal proc-esses suffi-cient 18 months or less. Controlled processes recommended 12 – 36 months. Robust proc-esses rec-ommended 24 months or more. Total cost

Total cost is the estimated total cost to acquire the software in order of magnitude. The value ranges of low, medium, and high in the table below are difficult to specify since these values may change over time and are only given as suggestions.

Minimal proc-esses suffi-cient

Low. Less than $50,000.

Controlled processes recommended Medium. $50,000 – $1,000,000. Robust proc-esses rec-ommended

High. More than $1,000,000.

Contractor development team size

The more people that are involved in the contractor’s develop-ment team the more communication is needed to coordinate the work, sort out misunderstandings, and resolve conflicts. The values in the table are not to be considered as absolute.

Minimal proc-esses suffi-cient

Controlled processes recommended 10 – 30 developers. Robust proc-esses rec-ommended

More than 30 developers.

3.2.3 Contracting environment

The contracting environment customization factors are related to the contractor and the organizational environment that the ac-quisition takes place in.

Organizations involved

The more organizations, such as stakeholders, supporting con-tractors, consultants, independent testing organizations, etc. are involved, the more communication is needed to coordinate the work.

Minimal proc-esses suffi-cient

Two parties involved in a simple dyadic relation-ship; customer and contractor.

Controlled processes recommended

More than two parties are involved; either sev-eral customers co-sourcing, or sevsev-eral contractors collaborating, or subcontractors are used, and possibly supporting contractors. The organiza-tional structure is not (too) complex.

Robust proc-esses rec-ommended

Several organizations are involved in a complex organizational structure composed of some or all of the following: multiple customers, multiple contractors, multiple subcontractors, multiple supporting contractors, etc.

3 Customization

© Daniel Svennberg 2001 27

Contractor development team characteristics

The contractor’s development team characteristics are the ex-perience, constitution, and performance of the contractor’s de-velopment team.

Minimal proc-esses suffi-cient

Low staff turnover. Mostly experienced and well-trained personnel. Well functioning teamwork. Efficient and appropriate working methods. High performance.

Controlled processes recommended

Moderate staff turnover. Some inexperienced and untrained personnel. Teamwork could be func-tioning better. Working methods need some im-provement. Medium performance.

Robust proc-esses rec-ommended

High staff turnover. Lots of inexperienced and untrained personnel. Teamwork not functioning optimally. Inefficient and inappropriate working methods. Low performance.

Experience with contractor

The type and level of experience with the contractor(s).

Minimal proc-esses suffi-cient

The customer and contractor have worked to-gether on several previous occasions and have a good relationship built on trust. The customer knows how the contractor works and vice versa.

Controlled processes recommended

First time working with a contractor with good results working for other customers, or mixed experiences with a previously used contractor.

Robust proc-esses rec-ommended

First time working with a contractor that is not well known or has had bad results working for other customers, or bad experiences with a previ-ously used contractor.

Geographic dispersion

Management becomes more difficult with customer and con-tractor on different geographical sites.

Minimal proc-esses suffi-cient

The customer, contractor, and other involved parties work at the same site or can visit and work on each other’s site on a daily basis.

Controlled processes recommended

The customer, contractor, and other involved parties work at different sites, with their respec-tive teams located at the same site, and cannot meet on a daily basis.

Robust proc-esses rec-ommended

The customer, contractor, and other involved parties have several teams involved in the project dispersed on several different sites and they can-not meet on a daily basis.

3.2.4 Other factors

The factors stated above are only suggestions and you will probably have to look at more factors to be able to customize the project appropriately. Other factors include: standards re-quired, laws and regulations, other projects, cultural differ-ences, application domain, system complexity, etc.

29

4 Roles

“No matter how much you want it to be a technical problem, it’s a people problem.”

– Unknown

Having good people and good teamwork is key to the success of the project. This chapter presents the different types of roles that may be necessary for a successful software acquisition pro-ject. One person can be assigned several roles. The roles de-scriptions are assembled from and based upon the different frameworks stated in appendix A, Marciniak and Reifer (1990), Salwin (1998), and the interviews with Urmi (2000), Oskarsson (2000), and Ran (2000).

4.1 Sponsor

roles

The sponsor roles are played by persons representing the or-ganizations that have the powers to initiate and cancel the ac-quisition project.

4.1.1 Sponsor

The sponsor initiates and supports the software acquisition pro-ject enthusiastically but also has the power to cancel it if it will cost more to finish than will be gained from finishing the pro-ject. The responsibilities of the project sponsor are:

• To define and communicate a goal and vision for the pro-ject.

• To provide a stable and adequate funding and set the fi-nancial limits and other bounds for the project.

• To appoint an acquisition manager that has the ultimate responsibility for the success of the project. The sponsor should be careful not to interfere too much with the way the acquisition manager manages the project.

4.1.2 Collaborating customer

Should the acquisition project be a co-sourcing project with several customer organizations involved, the collaborating

cus-tomer is another project sponsor whose collaboration needs to

be coordinated with the other sponsors.

4.1.3 Product manager

In some cases the software is part of a larger systems acquisi-tion with the product manager for that system playing the spon-sor role.

4.2 Managing

roles

The managing roles are played by the people on the customer side that are put in charge of managing, monitoring, and ad-ministering the proceedings of the acquisition project.

4.2.1 Acquisition manager

The acquisition manager is appointed by the sponsor and is ulti-mately responsible for the success of the project. Therefore, the acquisition manager has the final word on how the project should be performed. The tasks and responsibilities of the ac-quisition manager involves the following:

4 Roles

© Daniel Svennberg 2001 31

• To adapt and customize the acquisition strategy accord-ing to the project characteristics.

• To plan the project and execute it accordingly, re-plan as the project progresses, manage risks, and solve problems. • To recruit and organize people for the acquisition team,

perform teambuilding and training activities, praise and encourage people, smooth the path for everyone, and be supportive.

• To select contractor and supporting contractors.

• To negotiate and sign agreements with the contractor and supporting contractors.

• To manage the relationship with the contractor and other involved organizations built on moral, trust, communica-tion, and collaboration.

• To tell the contractor what the scope of the software is so that you and the contractor knows when you are done. • To manage the budget for the project within the limits set

by the sponsor.

• To coordinate the work for monitoring project progress and report to the project sponsor.

• To perform reality checks periodically. Take necessary ac-tions if the answers to the following quesac-tions aren’t satis-factory:

o Are we acquiring the right software? o Are we acquiring the software right?

o Is the contractor building the right software? o Is the contractor building the software right?

• To make necessary trade-offs between delivery time, total cost, product scope, and product quality if deviation from the previous estimates and specifications of them should occur.

• To coordinate work for evaluating and accepting the product.

• To prepare for post-contract support and maintenance of the product.

4.2.2 Technical manager

For large projects that are technically complex or with very high quality demands, such as life-critical software, the acquisition manager can delegate responsibilities for requirements and quality issues to a technical manager.

4.2.3 Contract manager

For projects with many involved organizations or projects

where the administrative burden of the contract is high, such as cost-reimbursement contracts, the acquisition manager can delegate responsibilities for contract management issues to a

contract manager.

4.2.4 Administrator

The administrative burden depends on the size and type of pro-ject. One should be careful not to over-administrate. Other members of the acquisition team perform the tasks of the ad-ministrator role, but for some projects a specific person can be assigned this role. The responsibilities of the administrator role include:

• To maintain configuration and change management on project documents such as contract, requirements specifi-cation, project plan, risk list, etc.

4 Roles

© Daniel Svennberg 2001 33

• To document and file correspondence, meeting minutes, and decisions.

• To administer payments to the contractor and supporting contractors.

• To document and file measurements on progress, quality, requirements changes, etc.

4.3 Assisting

roles

The assisting roles are played by different experts and support contractors that can assist and give advise to the managing, steering, and executing roles.

4.3.1 Acquisition expert

An acquisition expert can be used for giving advice on how to plan and perform the acquisition project, what contracting ve-hicle to select, and to train the acquisition staff in software ac-quisition management issues.

4.3.2 Technical expert

A technical expert can be used to evaluate the requirements and to give independent cost and size estimates. Furthermore, the technical expert can be used to review technical documents and to audit the technical knowledge and capabilities of the contrac-tor. The contractor can use technical experts for areas that they lack competence in.

4.3.3 Domain expert

A domain expert is not necessarily a technical expert but knows the field in which the software is intended to be used very well. The domain expert can therefore be used to develop and vali-date the requirements for the software. Another task for the domain expert can be to develop user training for the software.

4.3.4 Legal expert

A legal expert is essential for giving advise on how to write the contract and to inform on the actual laws that regulates the con-tract and other matters concerning the project, such as intellec-tual property rights, patents, licensing, warranties, copyright, etc. The legal expert can also help during disputes and schisms with the contractor. The use of a legal expert with software-specific knowledge is money well spent, especially in interna-tional projects, according to Salwin (1998).

4.3.5 Translator

A translator is, according to Salwin (1998), a role that can be played by a person that is good at translating layman’s terms into technical terms and vice versa. This can be useful in speci-fying the requirements and transferring the requirements to the technical staff of the contractor. This person should be skilled at translating the needs of the customer into something that the developers can use to build the system.

4.3.6 End user

The end user is the raison d’être of the software, unless of course the software doesn’t interact directly with a human user. There-fore, it is imperative to involve the end user on the elicitation of the requirements for the software, on evaluation of the graphi-cal user interface, and on evaluation of the functionality of the software product.

4.3.7 Maintenance and support staff

The maintenance and support staff has the primary tasks of keep-ing the software runnkeep-ing, makkeep-ing upgrades, fixkeep-ing bugs, and adding new features as well as giving technical support to the end users. Therefore, it is important to get suggestions from the maintenance and support staff on what to require from the software to make it easier to maintain and troubleshoot. They should also be involved in evaluating the maintenance and troubleshooting capabilities of the software and to review the

4 Roles

© Daniel Svennberg 2001 35

documentation to see if it conveys the necessary information required to do their jobs.

The maintenance and support staff should be involved early in the project in order to be able to discuss how the maintenance and support organization should be organized and to facilitate a smooth transition of the software once it is delivered to the maintenance and support organization with maintained con-figuration management. This role can either belong to the cus-tomer organization or be outsourced.

4.3.8 Independent verification and validation

An independent verification and validation contractor, IV&V, can be hired by the customer to make an in-depth technical assess-ment of the delivered software. This is mainly useful when the software affects peoples’ health or lives or when large amounts of money are at stake should the software malfunction. IV&V naturally increases the costs for the project significantly.

4.3.9 System engineering and technical assistance

When the buyer or seller lacks adequate resources or special-ized expertise, the use of a supplemental support or services contractor such as a system engineering and technical assistance contractor, SETA, can be necessary. The services provided by a SETA contractor can range from planning the project to testing, measuring, quality assurance, etc.

The use of SETA contractors should be considered for the fol-lowing areas, according to Marciniak and Reifer (1990):

• Technical risk.

• Independent testing. • Management support. • Integration.

4.4 Executing

roles

The executing roles are played by representatives from the con-tractors side, i.e. those developing the actual software product.

4.4.1 Contractor

The contractor can be a sole contractor or a prime contractor in case several collaborating contractors are used. Choose contrac-tor carefully – the contraccontrac-tor with the lowest bid or most opti-mistic schedule and budget estimates isn’t always the best

choice. No management practices can make up for having a bad contractor. Once selected, the contractor should be seen as a team member and not an adversary. The responsibilities of the contractor are:

• To develop and deliver the requested software. Should the project be cancelled, what has been developed up to that point should be delivered.

• To demonstrate that the sufficient technical and manage-rial capabilities exists to be able to develop the software. Should subcontractors or supporting contractors be used, their competence and capabilities needs to be demon-strated as well.

• To provide a feasible plan and work breakdown struc-ture for the project as well as realistic estimates on total cost and delivery time.

• To show the customer that progress is being made by holding regular demonstrations of the so far developed software.

• To make sure that the requirements are understood cor-rectly.

• To foster an effective, functional, supporting, and col-laborative culture in the relationship with the customer built on moral and trust. According to Ran (2000), both

4 Roles

© Daniel Svennberg 2001 37

parties must realize that they have a mutual responsibil-ity for the success of the project and should refrain from taking advantage of each other. The involved parties should work as a team, jointly solve problems, and re-frain from blaming each other.

• To promptly and openly inform the customer of unan-ticipated problems and schedule changes.

• To openly and jointly discuss, review, and mitigate risks.

4.4.2 Collaborating contractor

Should more than one contractor be used, the collaboration be-tween the contractors needs to be coordinated. This could be accomplished by assigning one of the contractors as a prime contractor with the responsibility to coordinate the work of the

collaborating contractors.

4.4.3 Subcontractor

The contractor can use subcontractors to deliver parts of the software. The competence and capabilities of the subcontractor needs to be evaluated, and you as a customer should have the right to decide which subcontractor should be used. If the con-tractor is using subconcon-tractors, make sure that they are in-volved as soon as possible in the project.

4.4.4 Supplier

It might be necessary to deal with a supplier of COTS and/or hardware during the course of the project.

5 Artifacts

“Everything has been said before, but since nobody listens we have to keep going back and beginning all over again.”

– A. Gide

In the acquisition processes different documents, reports, deliv-erables, and other artifacts are used. This chapter contains sug-gestions for the contents of those artifacts. The artifacts have been organized into the phase that they are most likely to ap-pear for the first time of the following four generic software ac-quisition project phases: inception, solicitation, monitoring, and finalization. The contents of the artifacts have been assembled from the different frameworks stated in appendix A unless oth-erwise stated.

Many of the artifacts are documents, and there are many rea-sons to document something, such as the following:

• To communicate vital information or to explain how you think.

• To help remember something.

• To help you think – the process of writing something down in a lucid and understandable way makes you do a lot of thinking you wouldn’t have done otherwise, re-gardless if the document is needed later or not.

5 Artifacts

© Daniel Svennberg 2001 39

• To legally prove that a transaction has occurred or that a commitment exists.

There is always a risk of documenting too much, writing things you won’t need later, or having too much bureaucracy. There-fore it is necessary for you to evaluate the need of each artifact and the applicability of each artifact checklist item in order to customize the artifact for the actual project. Some of the artifacts can be used as checklists during meetings while others require some written documentation. It is also important to remember that some of the artifacts, such as the acquisition plan, the re-quirements specification, and the risk list are likely to change during the course of the project. Therefore it is necessary to have some form of configuration and change management of these artifacts.

5.1 Inception

During the inception phase the needs of the software are identi-fied, the requirements are elicited, risks are analyzed, and the acquisition processes are planned.

5.1.1 Acquisition proposal

The purpose of the acquisition proposal is to present the needs for and the feasibility of the acquisition, and also what benefits and risks exists so that an informed decision can be made by the sponsor on whether to initiate the acquisition project or not. In addition to the frameworks stated in appendix A, this artifact is based upon Pressman (1997).

Contents checklist

‰Describe the organization’s needs for the software

prod-uct and how they are related to the organization’s objec-tives. What is the background of the needs? What is the current situation? What justifies the needs for the soft-ware?

‰Conceptualize the software product and give operational

scenarios. What will the software have done?

‰What are the short-term and long-term consequences for

acquiring the software? What are the advantages and drawbacks?

‰Analyze the alternatives for acquiring the software:

COTS, MOTS, internal development, outsourced devel-opment, joint venture develdevel-opment, enhancement of ex-isting software, or combinations of these alternatives. What are the risks, costs, and benefits for each alterna-tive? Also do a market study of the alternatives.

‰Describe whether the benefits, short-term or long-term,

outweigh the costs for acquiring the software?

‰Analyze the technical feasibility of the software. Are there

any developmental risks? Can the necessary technical re-sources such as skilled staff and equipment be made available? Does the technology exist to develop the soft-ware or is it better to wait?

‰Determine any infringement, violation, or liability that

could result from development of the system. Examine patents, copyrights, and other intellectual property rights.

‰Describe any external constraints on the project, such as

standards, regulations, other projects, interfacing systems, etc.

‰Describe any public relations issues. Is there value in

oth-ers knowing about this? How do we do that?

‰Describe how the project will be funded and give a time

frame for the project.

‰State any priorities, assumptions, limitations, and

trade-offs considered.

‰List potential contractors.

‰Give a motivated acquisition decision recommendation.

5.1.2 Project specification

deci-5 Artifacts

© Daniel Svennberg 2001 41

bounds, expected outcome, etc. which serves as a decision basis for the acquisition manager.

Contents checklist

‰Describe the organization’s needs.

‰Conceptualize the software product and give operational

scenarios. What will the software have done?

‰Define the purpose of the project. State the acquisition

project’s goal and vision in specific, measurable, accepted, realistic, and time-based terms. Specify the expected out-come of the project in terms of total cost, delivery time, product scope, and product quality, etc.

‰Specify the initially identified product requirements and

their relative priorities.

‰Describe any identified risks. ‰Specify funding and time bounds.

‰Identify and give instructions regarding interfaces and

collaborations with other projects.

‰Identify legal issues, such as intellectual property rights,

etc.

‰Specify any external constraints, such as standards,

regu-lations, interfacing systems, etc.

‰Give instructions on security, access to information, and

confidentiality issues.

‰List potential contractors and/or capabilities that can be

used for identifying potential contractors, such as re-quired competence, etc.

‰Describe routines for reporting project status. See the Project status report on page 56.

5.1.3 Acquisition plan

The purpose of the acquisition plan is to describe how the acqui-sition project will be managed.

Contents checklist

‰Describe the project’s background. Why is the project

done? What is its purpose? State the project’s objectives.

‰State the project funding and schedule bounds.

‰Define the terminology that will be used in the acquisition

project.

‰Describe what tools, techniques, and methods will be

used.

‰Describe any standards, laws, practices, and conventions

that are to be followed.

‰Determine problem resolution procedures.

‰Specify documentation requirements and procedures. ‰Specify configuration and change management

proce-dures.

‰Describe collaborations and interfaces with external

or-ganizations, support contractors, etc. and how the respon-sibilities are divided between the involved organizations.

‰Specify communication and reporting channels. List

im-portant addresses, etc.

‰Determine acquisition project team inter-group

coordina-tion and division of responsibilities.

‰Outline a master budget for the different processes. ‰Outline a master schedule describing the chosen

acquisi-tion life cycle and appropriate milestones.

‰Specify security, access to information and confidentiality

procedures.

‰Describe how public relations issues will be handled. ‰Describe routines for reporting project status

‰Describe how measurements will be collected, analyzed