Quantitative Goal Oriented Requirements Engineering

The GORE approaches described in previous section, such as the NFR and KAOS frameworks, have quantitative extensions [3,4,44,57, 64, 81, 126, 165,199, 222, 223,

265,270] that provide more precision for decision support than qualitative approaches

[103, 165]. In addition, quantitative GORE approaches help to clarify decision models

and evaluate the impact of decision choices on the stakeholders’ objectives.

The NFR framework and its variants (e.g i∗ [276, 278], TROPOS [47] and GRL [14])

have been quantitatively extended by replacing the “++/+” and “– –/–” symbols on the goal refinement contribution links with numbers (usually between -1 and 1), and attaching weight values (between 0 and 1) to the leaf goal/softgoal to denote the extent of satisfaction of the goal/softgoals [3, 4, 81,199, 270]. The degree of satisfaction of a goal/softgoal is computed using the weighted average of the degree of satisfaction of the sub-goals.

Giorgini et al. [103] developed a formal framework that uses NFR AND/OR refinements

to qualitatively and quantitatively reason about the goal model of a US car manufactur- ing company. The authors also presented a qualitative and quantitative axiomatisation for goal model primitives, and label propagation algorithms for each axiomatisation.

This approach does not capture uncertainty in decision models explicitly and lacks tool support for decision analysis.

Affleck et al. [3, 4] extended the NFR framework by proposing an optimisation model

that aims to maximise the attainment score (i.e. a score representing the degree of satisfaction of the softgoals); minimise denial of softgoals; and avoid superfluous im- plementation through the acceptance and rejection of leaf softgoals according to their scores and decomposition. Developing the optimisation model involves converting the NFR softgoal inter-dependency graph (SIG) to a directed graph with associated con- stants and variables, and the objective function is constructed from the graph constants and variables. The authors evaluated the optimisation model through simulation-based analysis of synthetic test data. While this approach aims at tackling the scalability issues in SIG, the optimisation model does not support elaboration of domain-specific decision models, and does not take into consideration the fact that real world problems have multiple objectives, for example, the model did not consider the cost of alternative operationalisations in determining the optimum solution; it analysed uncertainty in the graph variables through sensitivity analysis, which does not consider the probability of change in decisions that optimise stakeholder goals and probability of changes in model parameters’ values [166]; and finally, it lacks tool support for automated analysis.

Pasquale et al. [199] introduced an approach to quantitatively analyse security goals

and their trade-offs between other goals, such as availability and cost, for a service-based email system. Their approach consists of three phases, namely: modelling, formalisation and analysis phases. The modelling phase involves developing a threat model, goal model and asset model of the security concerns and goals of the system. The formalisation phase combines the models using mathematical representations of metric functions that quantify the satisfaction of the security goals and concerns. Finally, the analysis phase involves encoding the formalised model into a Boolean satisfiability problem, by adding a set of constraints to the model and then solving the problem using SMT Solver (Z3). This approach is semi-automated as the analysis phase requires manual encoding of the satisafiability problem in the solver, thereby requiring that decision makers have an understanding of how the solver operates. In addition, the authors compute goal satisfaction using point based estimate using range normalisation i.e. normalise the final output between -1 and 1.

Wei et al. [270] developed an approach, driven by quality requirements, for making implementation decisions according to the quality goals of a system, in the context of internet aware. The approach consist of four steps, namely: the model generation; sat- isficing status assignment; model reasoning and decision making. The model generation step involves transforming the goal models to a tree model and then to a symbolic for-

mular, according to the syntax of the proposed symbolic modelling language P. The

satisficing status assignment assigns quantitative (e.g. numbers in the range of 0 and 1) and qualitative (e.g. “++” and “– –” ) values to the generated symbolic formu- lar. The next step is to reason about the model in order to determine all candidate implementation decisions. Model reasoning can be quantitative, qualitative or hybrid of both; it is quantitative when the model’s contributions and satisficing statuses are fully quantified; it is qualitative when they are not quantified and hybrid when partially quantified. Finally, the implementation decision is made by incorporating users’ pref- erences and priority on goals. This approach, however, does not handle uncertainty in model parameter, and while it has tool support, there is still the issue of the scalability and complexity of goal models. The effect of this is that symbolic formula generated in step one becomes complex to read when dealing with large goal models.

Qin et al.[171] proposed a goal-based decision making framework that clearly demon-

strates the link between goal-oriented requirements engineering and multi-criteria de- cision making technique. Using a case study that involves selecting an IT solution for registering customer profiles, in the insurance domain; they evaluated their approach

by importing quantitative goal models developed in the jUCMNav tool [213] into Excel,

where strategies for alternative selection and/or elimination is implemented. The strate- gies used are disjunctive rules and Rank Order Centroid (ROC) formular. Disjuctive rule simply states that an alternative that exceeds certain threshold are jettisoned, whereas the ROC formular is used to convert the importance ranking assigned to different alter- native’s attributes (e.g., cost, scalability and interoperability) to quantitative weights. While this approach tackles decision making in a multi-objective optimisation context, it lacks an integrated tool support for analysing decision models and suffers from the scalability and readability problems in cases where the decision models are large. While the techniques described above enable the elaboration of domain-specific decision models in requirements and architecture decisions, the numbers (-1 or 1) assigned to

the contribution links in the goal models have no physical interpretations in the appli- cation domains [126]. In contrast, Letier et al. [165] proposed a quantitative extension

of the KAOS framework [65, 266] where levels of goals satisfaction are defined using

domain-specific metrics with physical interpretation. They developed a technique that specifies partial degrees of goal satisfaction, and quantifies the impact of alternatives on the extent of goal satisfaction in terms of refinement equations, which are defined over random variables involved in the system‘s functional goals. They computed their ob- jective functions for higher-level goals using estimated probability distribution functions from the leaf or low level quality variables. This approach does not cater for automated analysis of goal models, as the analysis is done through an ad-hoc process that involves the use of analytical and numerical methods.

Heaven et al. [126] leverages the quantitative goal refinement model based on the KAOS

framework presented in [165] to simulate and optimise the impact of alternative system

designs on high level goals using multi-objective genetic algorithm (NSGA2). They found the Pareto optimal design options among the alternatives options that optimises the achievement of 8 and 14 minutes response time of the London Ambulance System, at a low cost. This approach, however, lacks tool support and requires manual encoding of the simulation models in a general programming language, such as R and MATLAB.

Sabetzadeh et al. [222] proposed a goal based approach, using the KAOS framework, for

assessing the satisfaction of new technology goals, such as safety and reliability goals. The approach uses sensitivity analysis to figure out system components that needs fur- ther improvements in satisfying the system goals. This approach involves the following steps: (1) developing a goal model of the system; (2) plan and collate evidences in order to quantify the probabilities that low level goals are satisfied, probabilities of obstacle blocking low level goals and probabilities of risk occurring due to incomplete decom- position of goals; (3) Elicit the probabilities from experts according to the evidences presented; (4) propagate the elicited probabilities from the leaf goals to compute the probability distributions for the satisfaction of the high level goals; (5) given that un- certainty about the satisfaction of the overall goal exist, perform sensitivity analysis to determine which model parameter inputs lead to great variations in the model output

parameters. Although this approach has tool support that is presented in [223], it is

only targeted at safety and reliability technological quantification and the complexity of goal models limits the scalability of this approach.

In summary, Section 2.2 describes existing qualitative and quantitative Goal-oriented requirement engineering (GORE) approaches used in making design decisions in Re- quirements engineering. Qualitative goal oriented approaches such as the NFR frame- work [180], i∗/TROPOS [47,276], URN/GRL [14] and KAOS [264] allow one to model, reason and analyse the impact of decisions on stakeholders’ goals. However, they only provide information about how the decisions positively or negatively impact stakehold- ers’ goals. They provide no quantitative information about the impact of alternatives on goals. Quantitative goal oriented approaches, however, are more accurate in modelling, reasoning and analysing domain-specific decision problems [3,199] [126,165]. But they require higher modelling effort and generally have limited automated tool support for decision analysis.