Cost Bene…t Analysis Method (CBAM)

additional architectural approaches, risks, sensitivity points and trade-o¤

points.

4. Reporting

Present results. In this step the results and collected information from the previous steps of ATAM (approaches, scenarios, attribute-speci…c ques-tions, the utility tree, risks, non-risks, sensitivity points and trade o¤s) are collected and put into an easily understandable format to be presented to the decision maker or similar.

9.2 Cost Bene…t Analysis Method (CBAM)

CBAM is an extension of ATAM that takes both the architectural and the eco-nomic implications of decisions into consideration when evaluating alternative architectures. CBAM focuses on how an organisation should invest its resources to maximise gains and minimise risks and o¤ers a set of techniques for assessing the uncertainty of the judgments involved in assessing costs and bene…ts for each alternative architecture.

CBAM begins where ATAM concludes and depends on the artefacts produced by ATAM. ATAM uncovers the architectural decisions made and links these to busi-ness goals and quality attribute response measures. CBAM builds on these pieces of information by eliciting the costs and bene…ts associated with the decisions.

As the architectural solutions have both technical and economic implications the business goals of a software system should in‡uence the architectural solution used by software architects or designers. The technical implications are the char-acteristics of the software system (namely the quality attributes) while the direct

…nancial implications are the cost of implementing the system. However, the quality attributes also have …nancial implications as the bene…ts are measured as

…nancial return on investment derived from the system.

When the ATAM is applied to a software system the result is a set of documented artefacts. These are the following:

A description of the business goals that are critical for the success of the system.

A set of architectural views that document the existing or proposed architec-tures.

A utility tree that represents a decomposition of the quality goals of the stake-holders for each architecture that starts with high-level statements of quality attributes and ends with speci…c scenarios.

72 9. Architecture/Design trade-o¤ analysis

A set of architectural risks that have been identi…ed.

A set of sensitivity points, which are architectural decisions that a¤ect some quality attribute measure of concern.

A set of trade-o¤ points, which are architectural decisions that either positively or negatively a¤ect more than one quality attribute measure.

ATAM also identi…es the set of key architectural decisions that are relevant to the quality attribute scenarios elicited from the stakeholders. These decisions result in speci…c quality attribute responses, such as levels of availability, performance, security, usability and modi…ability. Each decision also has associated costs. This means that using redundant hardware to achieve a desired level of availability has one cost and check-pointing to a disk …le has another cost. Furthermore, both architectural solutions will result in (presumably di¤erent) measurable levels of availability. It is these …nancial considerations that CBAM addresses.

CBAM is performed in two iterations: (1) Establish an initial ranking and (2) Incorporating uncertainty. An overview of the two iterations of CBAM is given below.

Iteration I: Establish an initial ranking

In the …rst iteration of CBAM a series of 9 steps are executed to establish an initial ranking of the result from ATAM. This intial ranking is later re…ned in the second iteration of CBAM. These steps serve to reduce the size of the decision space, re…ne the scenarios, collect su¢ cient information for decision making and to establish an initial ranking of architectural strategies derived using ATAM.

Step 1: Collate scenarios. This step collates the scenarios elicited during the ATAM exercise and asks stakeholders to contribute with new scenarios if such exist. Then the scenarios are prioritised in relation to satisfying the business goals of the system and the top one-third of the scenarios are chosen for further study.

Step 2: Re…ne scenarios. This step re…nes the scenarios from Step 1 by focusing on their stimulus/response measures. Then for each scenario the worst, current, de-sired and best-case quality attribute response level is identi…ed and documented.

Step 3: Prioritise scenarios. In this step 100 votes are allocated to each stake-holder and each of these are asked to distribute the votes among the scenarios by considering the desired response value for each scenario. The votes are then summed and the top 50% of the scenarios are chosen for further analysis. This is done by assigning a weight of 1.0 to the highest rated scenario and then rating the other scenarios in relation to this scenario. The result of this voting is the scenario weights used to calculate the overall bene…t of an architectural solution.

9.2 Cost Bene…t Analysis Method (CBAM) 73 Step 4: Assign utility. In this step the utility for each quality attribute response level (worst-case, current, desired or best-case) is assigned to all scenarios. The quality attributes of concern in this context are those identi…ed in the previous step.

Step 5: Develop architectural strategies for scenarios and determine their expected quality attribute response levels. In this step the architectural strategies address-ing the top 50% from Step 3 is developed further to determine the expected quality attribute response levels that will result from implementing the di¤erent architectural strategies. Given that an architectural strategy may a¤ect multiple scenarios, this calculation must be performed for each a¤ected scenario.

Step 6: Determine the utility of the expected quality attribute response level by interpolation. In this step the elicited utility values is used to determine the utility of the expected quality attribute response level. The calculation is performed for each a¤ected scenario.

Step 7: Calculate the total bene…t obtained from an architectural strategy. In this step the utility value of the “current” level is subtracted from the “expected”

level and normalised using the votes from Step 3. Then the bene…t of a partic-ular architectural strategy is summed over all scenarios and all relevant quality attributes.

Step 8: Choose architectural strategies based on Return On Investment (ROI) subject to cost and schedule constraints. In this step the cost and schedule im-plications of each architectural strategy is determined. Then the ROI value for all remaining architectural strategies are calculated as a ratio of bene…t to cost and ranked according to their ROI values. These architectural strategies are then selected from the top of the rank list and down until the budget or schedule is exhausted.

Step 9: Con…rm the results with intuition. In this step the chosen architectural strategies are evaluated to examine if they seem to align with the business goals of the organisation. If that is not the case, issues that may have been overlooked during the analysis should be taken into consideration and reiteration of some of the previous steps might be necessary. If signi…cant issues exist it might be necessary to reiterative all nine steps.

Iteration II: Incorporating uncertainty

A more sophisticated and realistic version of CBAM are achieved by expanding on the steps of CBAM iteration 1. This is done by incorporating uncertainty.

This version of CBAM is covered by iteration 2 of CBAM. When incorporating uncertainty information about risk estimation and uncertainty and the alloca-tion of development resources are added into the result from iteraalloca-tion 1. Each category of relevant information may potentially a¤ect the investment decisions

74 9. Architecture/Design trade-o¤ analysis

under consideration. Therefore, the way each of the nine steps of iteration 1 is augmented must be considered carefully for correctness and for practicality.

The stakeholder weights in iteration 2 of CBAM are assigned by voting using utility scores. Utility is measured as the relationship between a score and the best score possible, which is usually set to equal 100%. Utility scores are then assigned to each architectural scenario from iteration 1 given in terms of worst case, current, desired and best. Utility scores are given to both scenarios and architectural strategies. To eliminate and prioritise scenarios the stakeholders vote individually or together and the total sum of votes for all scenarios is normalised over 100%. This is to force the stakeholders to prioritise among the alternatives and to avoid them giving votes to an architectural strategy independent of the other architectural strategies involved.

The AORDD security solution design trade-o¤ analysis described in Part 4 is a security speci…c design trade-o¤ analysis. This trade-o¤ analysis incorporates ideas from both ATAM and CBAM but is extended to include security solution and misuse-speci…c parameters in addition to the economic implications from CBAM and other relevant development, project and …nancial perspectives as input to the trade-o¤ analysis. The analysis is also extended to incorporate the notion of operational security level, as discussed in Littlewood et al. (1993) [74], to evaluate the operational security level of an information system.

Part III