Spiral 2 Evaluation

have many components matching the case study criteria. Sites III and IV are aimed at encouraging open source development. Their projects may not be stable, but there are good options for reuse as interfaces are accessible. There are a large number of projects, varying in maturity and level of documentation. Site V mainly provides standalone applications, so little information about integrating the software is available. The site does offer a large number of titles and the available software may prove suitable for integration into other component-based systems.

5.6 Spiral 2 Evaluation

The second Spiral of the investigation addressed RE2 in developing a process for com-ponent selection. To evaluate the Spiral, the stakeholder Win conditions (Table 5.2) and Spiral goals (Table 5.1) are now discussed. This evaluation is based on the work of Spiral 2, although very relevant work took place in Spirals 3 to 6. That work, and its impact, will be discussed in Section 5.8.

One issue with existing processes for component selection is a lack of uptake (Li et al, 2006) which may be improved by providing simpler processes: better indicating the benefits or flexibility to integrate with existing work practices. This Process, developed with consideration of these issues, is adaptable to allow for alternative implementations of Steps to match the local development environment.

The Win conditions for application developers are to have an intuitive process with justification and low overheads. The Process itself was developed to take the basic steps of a generic selection process. It is able to be followed manually, while also providing options for automation. At each Step, documentation is created which can later be used to justify the selection. This is also a win condition for quality assurance. The final win condition for the application developers is that the Process has low overheads, with the main overhead for using the Process at Spiral 2 in providing the ideal specification. As this is a formalisation of defining the requirements, the overhead can be considered low.

Component developers are able to see clearly how their components are assessed through the Process and the use of the ideal specification. The Process requires infor-mation from repositories at the shortlisting stage (Step 2) and in the downloading of executables and documentation at Step 4. Brokers can assist integration of selection

CHAPTER 5. THE CDCE PROCESS

processes by providing search facilities with the required fields and/or XML (or similar) output of the repository data. Also of interest to brokers is that fully-functional versions of the software would need to be provided for evaluation.

The academic Win conditions include the need for a justifiable, repeatable process, which has already been discussed. In addition, the low coupling on Steps and the use of XML for documentation, inputs and outputs will assist automation. The structure of the CdCE Process has been developed with reference to processes described in the literature and includes the phases of filter, evaluate and selection. The outcomes of this Spiral were peer reviewed and presented at the Asia-Pacific Software Engineering Conference (APSEC) (Maxville et al, 2003b) and at the Postgraduate Electrical Engineering and Computing Symposium (PEECS) (Maxville et al, 2003a).

The preceding review supports the view that the stakeholder Win conditions have been met.

Goal 2A Focus Quality: Produce a structured, repeatable process for selecting and evaluating components

Viewpoint Quality Assurance personnel

Q2A1 Is the process defined according to accepted standards? YES Goal 2B Focus Usability: Consider existing processes and

organisa-tional requirements, provide tools and automation Viewpoint Application developer

Q2B1 Can the process interface with organisa-tional/development processes?

PART Q2B2 Is the process easy to understand and use? YES Q2B3 Has been tested on real world examples? PART Goal 2C Focus Intelligence: Identify areas where intelligence and

au-tomation can be applied Viewpoint Application developer

Q2C1 Have areas for automation been identified? YES Table 5.8: GQM Summary - Spiral 2 (Part 1/2)

The following discussion refers to the goals listed in Tables 5.8 and 5.9. Each goal is included as a heading followed by a discussion relating to that goal. Many of the goals were further addressed by work in later Spirals.

5.6. SPIRAL 2 EVALUATION

SPIRAL 2 Purpose Evaluate Result

Issue effectiveness of

Object the developed selection process Context Spiral 2

Goal 2D Focus Innovation: Consider novel approaches to the process Viewpoint Academia

Q2D1 Is there anything novel in the process? YES Goal 2E Focus Dynamics: Allow for dynamic assessment with context

Viewpoint Application developer

Q2E1 Does the process allow for iteration? YES

Q2E2 Context is included in the selection? YES

Goal 2F Focus Reuse: Where possible make use of existing code and artefacts

Viewpoint Application developer

Q2F1 Has the work reused external resources? PART Table 5.9: GQM Summary - Spiral 2 (Part 2/2)

Quality

The outcomes of this Spiral adhere to current standards for documenting processes, with the CdCE Process drawn as a UML activity diagram. The description includes input and outputs for each Step. Further details on the implementation of the Steps and the strategies that have been developed are in subsequent chapters.

Usability

An example of the process integrating with the system is shown in Section 5.3. To better address this question, the process would need to be trialled in development environments, which was out of the scope of this project.

The Process is defined in an easily understood form - the activity diagram - and a top-level understanding is easily attainable. The Process can be used manually to encourage better structure for selection, or with automation to allow for larger repositories and more complete documentation.

The initial cases worked through were real world examples, with manual data col-lection from a variety of sites. The case study in Section 5.5 shows the process applied to assessing real software from public repositories. For full satisfaction of this question, the Process would need to be trialled in a software development environment. Thus it is considered to be partially satisfied.

CHAPTER 5. THE CDCE PROCESS

Intelligence

The areas identified for automated support include: 1) automated documentation in XML between steps, 2) automated test generation, 3) an automated test oracle and 4) automation to support selection based on a set of criteria. These were expected to be able to include some machine intelligence as an outcome of future Spirals.

Innovation

The Process includes the basic steps for a dynamic evaluation of components. These generic steps allow for flexibility to explore various strategies and implementations in this project. It also allows localisation of the Process to match and evolve with organisa-tional standards and contexts. This detachment from any particular implementation is novel and is distinct from the decision support approach to choosing a selection process described in Mohamed et al (2007b).

The Process separates the test generation from the shortlisting. The tests are gener-ated from the requirements, allowing testing to be truly comparative as the tests are then adapted and executed for all candidates. The functional specification is encapsulated in the Z specification, which also provides the contextual information for testing. Together, the approach to test generation and the use of Z notation is novel in the component selection literature.

The concept of the ideal specification is also novel. It provides a standardised form to record the requirements, based on the work of Spiral 1. The schema can be used for describing all components of interest and the requirements for the selection.

Dynamics

As the selection process is interdependent with the wider software development, and with the suitability of available candidates, it is important to be able to adapt to each of these inputs. At the wider process level, the repeatability of the CdCE Process supports revisiting the selection task within a project. The manual case study utilised a procedure of progressively adding selection criteria to filter for candidates. This iteration allowed the selector the flexibility to respond to the available candidates, which may not fully match the ideal specification. It is also possible to re-specify the requirements after shortlisting, which is indicated by the loop back between Step 2 and Step 1.