After applying the model repeatedly to several software development projects following the same approach presented in this research, the data repository will increase and become more stable and reliable. Accordingly, the project manager can start using the model as an important support tool for building future decisions on QA plans. Back to our project details component shown in Figure 6.3, the project manager, after entering the new project details as usual, will click on the scenario button to get some informed estimates on the expected outcome of proposed QA activities. By clicking on the scenario button, a new window will pop up that shows the following details:
example is High.
• The coverage weight value given to the work product (α) out of the total phase size.
• The size of the phase artifact and the actual size of the work product.
• The estimated number of defects injected into the work product according to the defect injection rate and the actual size.
• The defect escalation factor and the labour rate values.
An overview of scenario windows is depicted in Figure 6.9.
Figure 6.9: Scenario Making Component
Having all relevant details ready for the chosen work product, the project manager can choose a single practice or a group of QA practices from the listed available techniques to find out the possible outcome of the proposed defect detection and removal plan (Figure 6.10).
The choice of QA practices is subject to the project manager’s insight and to the constraints associated with the project development process, the availability of testers fa- miliar with an individual practice, the readiness of the practices, etc. Suppose the project
manager chose to apply two QA practice to the work product High, a new form would appear with empty boxes which would require the weight value of each one of the two QA practices chosen. Along with weight input boxes, relevant experience details related to the two QA practices such as DRE, execution time and defect removal costs are auto- matically retrieved from the system repository. As discussed before, experience details are determined from previous software development projects.
Figure 6.10: Estimated Result of the QA activity
The next step after the weights determination is to get an overview of the expected outcome of the current QA practices with respect to the two practices chosen and the weights assigned to them. The system will perform the needed calculations following
the equations discussed earlier in our formal model, and produce the result in tabular form. The results are detailed for every QA practice chosen such as the estimated number of removed and escaped defects, the execution cost and duration it requires, the overall defects removal cost and so on. The Unspecified column considers the situations in which part of the work product is left uninspected. Accordingly, the system will calculate all aspects of cost related to that part of the work product.
In Figure 6.11, the button labeled "Step 3" is the optimisation step of the system whereby an optimal solution is generated according to sets of conditions. Such condi- tions are the defect removal efficiency (DRE) value of the overall work product under inspection, the overall execution cost, removal cost, etc. The project manager can adjust the conditions (on the right of the figure) by entering their values and generating the so- lution. This solution will find the optimal distribution of weights that should be assigned to the two QA practices chosen at the beginning. The result of the optimal solution found will automatically change the values of weights given to the QA practices chosen.
6.5
Summary
The decision-making process regarding QA activities is subject to a lot of ongoing debate and different views due to the lack of a knowledge-based system to distinguish decision alternatives from each other. This chapter presented the implementation of our QA system tool and defined the main components of it. An outline was given to the formal steps of data entry and data analysis procedures. Also, this chapter showed how to utilise the optimisation function to find optimal solutions through the defect removal efficiency (DRE) required or on a cost of execution basis. The next chapter evaluates and assesses the functionality of our model and demonstrates its ability to make the trade-off process between QA plans on the basis of quality, cost and time.
Evaluation of the System
Objectives
• To simulate the system functionality.
• To apply a hypothetical case study to our system to demonstrate its functionality.
• To signify the system’s role as a decision support tool.
• To evaluate the efficiency of our model.
7.1
Introduction
The evolving nature of our system necessitates that the system should be piloted within a software development organisation to a few projects to build up its repository before it can be used as a decision support tool. However, taking into consideration the limited time allotted to do our research, it was difficult for us to implement our system for several projects developed in a single organisation in order to calibrate it within a specific do- main of software projects. Moreover, because mid-sized and large software development
projects, which our model mainly targets, last on average from 8 months to 1 year in terms of schedule, it was too difficult for us to conduct the required evaluation during our PhD research period.
In our endeavour to get some ready and real data to feed into our system, some connec- tions were made with organisations that have software verification and validation activi- ties like NASA’s1independent verification and validation department, and online software engineering warehouses like the Promisdata2 repository. However, they responded that they had not got any defect data which conformed to our risk-based approach of work product categorisation.
Although this response was frustrating, it was at the same time positive because it showed the importance of our model and emphasized the originality of our work as none of those leading defects warehouses have such data available nor have they implemented a similar quality management system.
Another contact was made with Software Migrations Limited (SML), which is a UK- based software transformation company, to pilot our system in their software transforma- tion projects. It is expected to start implementing our model with them after the comple- tion of this research. The implementation of our system will be an additional service in a way that guarantees that our system will not affect their current QA activities.
In this chapter, fictitious data is used which depicts data of real QA activities of soft- ware development projects. This data will be used as an input to our system to simulate its functionality as an informative decision support tool for evaluating potential QA plans.
1NASA has an independent Verification & Validation program responsible for implementing QA best
practices for complex software systems.