In summary, this chapter has defined the concepts of software artefacts, traceability, change impact analysis, consistency, consistency checking, consistency management and change propagation. This serves as a preamble to discussing related work and the proposed consistency management approach in Chapter 3 and 4, respectively.
3
CHAPTER
THREE
LITERATURE
REVIEW
This chapter explores related work relevant to artefact consistency management. The approaches, which span a number of research areas, are first classified. Following a detailed discussion of their characteristics, an evaluation is carried out to identify their benefits and shortcomings. This leads to establishing the characteristics of a potential solution in Chapter 4 addressing the identified shortcomings.
3.1
Introduction
State-of-the art solutions presented in this chapter contribute to advancing the field either by considering the problem in a holistic manner or by focusing on specific aspects of it. It is not within the scope of this work to outline the history of all related fields and to identify the main challenges of the individual research areas. Instead, the aim of the analysis is to compare solutions originating from diverse research areas based on common characteristics related to the discussion of artefact consistency management and to reveal the extent to which they advance artefact consistency management.
Specifically, the focus of the review is to identify to what degree the issue has been addressed and which research areas contribute to solving it. To obtain answers to these questions, relevant solutions were analysed and evaluated based on the questions described in Subsection 3.2.4. The rest of the chapter is organised as follows. Firstly, the methodology used to carry out the review is presented in Section 3.2, followed by a classification (Section 3.3 and analysis of solutions (Section 3.4). The chapter concludes with the outcomes of the evaluation discussed in Section 3.5.
3.2
Methodology
The review process was guided by principles of systematic literature reviews in software engineering, based on a description of a methodology by Biolchini et al. [59] and a survey provided by Kitchenham et al. [60]. The aim of adopting a non ad hoc approach was to allow the reproducibility of the investigation. It is also worth noting, however, that despite the up front planning, the review process is an iterative one. Therefore, certain planned aspects of it, such as search terms and inclusion criteria were subject to change.
3.2.1
Planning
3.2.1.1 Research Questions
The following research questions are formulated to provide the groundwork for conducting the survey.
RQ1. What research areas contribute to artefact consistency management?
RQ2. Which aspects of the problem of artefact consistency management do these approaches address?
RQ3. Do these approaches implement the proposed solution and what level of automation do they provide?
RQ4. Which software artefacts do these solutions cater for?
3.2.1.2 Search Process
Firstly, Google, Google Scholar1and CiteSeerX2were selected to perform search. Secondly, the types of information sources were considered: conference proceedings, journals, PhD theses and websites dedicated to solutions. Finally, search terms were identified, which include: software traceability, traceability creation, traceability maintenance, software artefact, software change impact analysis, software artefact consistency, software change propagation, heterogeneous soft- ware artefacts, software change management, software configuration management, consistency
checking.
Inclusion and Exclusion Criteria
At the outset, studies originating between 2000 and 2013 were selected to identify the latest results and approaches in each area. This was later extended to studies dating back prior to 2000
1https://scholar.google.co.uk/ 2http://citeseerx.ist.psu.edu/index
to include works laying the foundations of these areas.
The above search terms also returned not directly relevant results, primarily due to two reasons. Some concepts, such asconsistencyorconsistency checking, appear in various research areas. To filter such results and exclude irrelevant ones, specific search techniques were utilised, such as the exclusion search operator.
Besides, some of the search terms refer to large research areas, therefore studies directly relevant to artefact consistency management and the research questions had to be selected. To illustrate this process, the example of the term software traceability is taken. Firstly, the relevance of traceability in artefact consistency management is identified. Since traceability allows connections between heterogeneous software artefacts to be established, traceability solutions aimed at supporting the automatic linking of artefacts are considered. As software artefacts evolve, their connections are also impacted, therefore traceability maintenance solutions are also within the scope of potential solutions. Secondly, further questions were raised to select relevant solutions. For example:
• Are there any approaches that incorporate traceability to support software evolution? • Are there any standalone traceability solutions that cater for heterogeneous software artefacts? A similar process was applied with respect to each search term pertaining to larger research areas.
3.2.2
Execution
The original study was first carried out between 2012 and 2013 and was followed up periodically to identify unseen work. This approach proved to be effective as further examples of relevant work were found after 2013 until the submission of this thesis.
Data Collection
The following information was extracted from the studies in question: • The aim and scope of the work
• Motivation
• Supported software artefacts and extensibility
• Level of automation: is the solution manual, or semi-automatic, or fully automated? Which aspects of the solution are automated, if any?
• Implementation details of prototype, if applicable, with respect to extraction and storage of software artefacts and trace links.
• Stages of artefact consistency management (introduced in Chapter 4) supported.
3.2.3
Results Analysis
Based on the findings of the search process, a classification of the investigated studies was established. The main premise of the classification is that some solutions are holistic and support multiple aspects of artefact consistency management, while others focus on solving a particular aspect of the problem. Since the boundary between the specific research areas is often blurred, the classification presents a major challenge in this review process.
Besides the categorisation, the findings also allow an evaluation of the individual solutions to be carried out, the criteria and results of which are discussed in Section 3.5.
3.2.4
Related Surveys
To the best of our knowledge, a comprehensive overview combining and assessing results achieved in areas that contribute to artefact consistency management, has not yet been produced. Closely related to this survey is a review of traceability in requirements and model-driven development by Winkler et al. [36]. The classification approach they follow is based on grouping solutions around the hindering factors traceability practices face and strive to overcome. These are namely natural, technical, economic and social limitations. The authors classify research publications based on the artefacts among which traceability is established.
In "A Review of Software Change Impact Analysis", Lehnert presents the results of an investigation of change impact analysis and identifies five scopes of impact analysis [42]. Accordingly, impact analysis can be performed at the source code,architectural model, and
requirements modellevel. Additionally, some solutions considermiscellaneous artefacts, such as
documentation, configuration files, bug trackers. Finally,combined scope approachesprovide a comprehensive analysis to trace impacts across different types of artefacts. Some of the solutions mentioned in Lehnert’s survey are highlighted in this work for their contribution to artefact consistency management.
Sun et al. provide a survey of code-based change impact analysis techniques proposed between 1997 and 2010 in [39]. Their work constructs a framework to characterise and classify these techniques based on a set of criteria including the impact set, the type of analysis, intermediate representations, language support, and tool support. While this work is specific to source code artefacts, which is the focus of the majority of change impact analysis solutions, it provides useful information on the evaluation of source code change impact analysis techniques, which
may be extensible to other artefacts.
Although not a survey paper, a study by Cleland-Huang et al. [61] presents a brief summary of work in the areas of trace creation, trace maintenance and trace integrity, the correctness of identified and maintained links. This work contributes to assessing current solutions. Similarly, Gotel and Finkelstein investigate the requirements traceability problem and reveal that one of the reasons contributing to the persistence of the issue is the lack of a common definition of
requirements traceabilityand related concepts [26]. The authors also provide a framework for
addressing the problem.
3.3
Classification
In this section, existing approaches are classified, which is followed by a detailed review in Section 3.4. As shown in Figure 3.1, solutions can be grouped into two primary categories based on the scope of the problem they aim to address. Holistic solutionsare characterised by a comprehensive support for managing artefacts incorporating techniques from traceability, change impact analysis and various fields of software change management including configuration management. Conversely, specific solutionsfocus on techniques for the individual aspects - linking artefacts, assessing the ripple effects of changes and controlling the evolution of artefacts by various means.