Cost of Software Quality (CoSQ)

The term CoSQ, which refers to the cost of software quality, describes the trade-off mech- anism between delivering software of a high or an acceptable level of quality and the cost associated with it [88, 92]. It is a mechanism that helps software project managers de- termine the accepted and affordable level of quality for their products. Moreover, CoSQ helps QA practitioners find an answer to the inevitable question raised by software project stakeholders about the cost savings that would be gained that would offset any expendi- tures on quality improvement [133]. Also, it helps software project managers to determine

the possible alternatives of quality improvements practices [64] and to make a comparison between them on a well-defined basis. Generally, there is no validated and agreed-upon cost of software quality model today. Most software organisations use a quality cost model taken from manufacturing and adapted to be applicable to software. This model classifies the cost of quality into the cost of achieving quality and the cost of poor quality [150,143,88,92].

3.2.1

CoSQ models

The most well-known cost quality model in the literature is what is called the PAF model [122, 88, 131, 72]. The PAF model was originally used in manufacturing industry and then was adapted by software engineers to be applicable to measure the cost of quality activities in software development.

Figure 3.1: PAF Model Philosophy

According to the PAF model, as seen in Figure 3.1, the cost of quality is measured as the sum of conformance and non-conformance costs. Conformance cost, which is divided into prevention and appraisal costs, is the cost related to implementing practices and techniques to prevent poor quality. For example, implementing pro- cess improvement models like CMMI [37] or using code generator tools to help

minimise the introduction of errors are two kinds of prevention costs. On the other hand, the cost of applying formal inspection for software project deliverables or the cost of document reviews are considered to be appraisal costs as they prevent de- fects from propagating to later phases. Non-conformance cost is the cost of quality failure or the cost of not conforming to the original requirements of the software. Such costs can be either before (internal) or after (external) the software release. The main assumption of the PAF model is that investing in conformance costs will pay off in reducing the cost of failure, and that investing in prevention activities will reduce appraisal costs [131]. In other words, there is an inverse relationship between conformance and non-conformance costs, therefore balancing this rela- tionship between the two in a way to make it optimal is crucial to any software development process.

2. Crosby Model

Similar to the PAF model, the Crosby model considers the cost of software qual- ity as the price invested to conform to requirements and the price paid for non- conformance to requirements [131,72]. The main concept of the Crosby model is that doing the job right the first time is cheaper than doing it later.

3. Opportunity and Intangible Cost Model

This model was successfully integrated into the original PAF model by Sandoval- Chavez and Beruvides [130, 131]. The model quantifies the cost of losing oppor- tunities and unrealised profit due to a reduction of customers and competition in markets. It argues that any lost profit that could have been achieved by increasing quality is to be considered as part of the total quality cost along with the confor- mance and non-conformance costs.

It would seem that a lot of features of the current CoSQ model are derived from or follow the same approach as the PAF model which divides the total cost into conformance and non-conformance costs. As is shown in Figure 3.2, which il- lustrates the relationship between the conformance and non-conformance costs, as the software quality increases by investing in appraisal and prevention costs, the non-conformance cost decreases. However, at some point during the software de- velopment process, the conformance cost increases rapidly, compared with a slight decrease in the non-conformance cost. The reason is that some errors may have a limited impact on the software compared to a significant loss in the time and effort required for finding and fixing them. A senior manager in one of the leading soft- ware development organisations stated that " I’d rather have it wrong than have it late, we can always fix it later." [116]. This supports the idea that balancing confor- mance and non-conformance costs is more cost-effective than trying to eliminate the non-conformance cost completely [117].

3.2.2

Industrial Data of CoSQ

In the previous section, a theoretical overview was given of the cost of software quality (CoSQ) and the models used to quantify it. This section presents some empirical data from the literature on the percentage that the cost of software quality represents out of the overall software development effort. According to a study conducted by the Price Waterhouse Coopers [123], which included a survey of 19 UK software development organisations, conformance cost (appraisal and prevention) make up 23-34% of the total software development cost. Moreover, non-conformance cost, that is the cost of internal and external failures, is weighted at about 41% of the total development cost. Another

Figure 3.2: CoSQ Model : Source [131]

study showed that conformance cost consumes 40% of the overall cost of quality[134]. With regard to a specific quality technique, Votta [145], in his experiment, found that formal inspection, which is a typical defect prevention practice, consumes about 10% of the software development time.