Measuring techniques

The main focus of measuring techniques is the demands to know the QAs emanating from the architectural structures. The results derived from these techniques are usually represented in the form of numerical values. They are generated in response to quality issues raised by the various stakeholders. These techniques are only able to answer a few questions revolving around quality attributes such as performance and modifiability. Due to the various complex- ity issues which surround measuring techniques, understandability matters often arise amongst stakeholders. Examples of measuring techniques include simulations, metrics, prototypes, and experiments,Zayaraz[2010].

The following subsections explain important concepts related to measuring methods:

2.5.2.2.1 Metrics: is “a system of weights and measures to express a hundred times the unit” Simpson et al. [1989]. Also, it is defined as ”a system of weights and measures based on the principle that each quantity should have one unit”,Walker[1988].

On a particular observable measurement of the architecture, the placed quantitative inter- pretation is called metrics. Fan in/fan out of components is an example of metrics. Answers to overall complexity that suggest locations that likely have to change or where change might be difficult, can be provided by the most well-researched measuring techniques.

The following discussion represents a set of metrics (as an example), in order to measure complexity and therefore predict areas of change in a real-time telecommunications system built, using object-oriented design (Arora, 1995), cited inClements et al. [2002a]). As opposed to code, examining a detailed design can produce more information which makes the following metrics more appealing.

• Number of component clusters – An object that composes the number of component clus- ters it contains. For example, a computer is composed of a screen, keyboard, software, etc., and other objects,

• Depth of structure: An object that is defined by the number of layers of encapsulation. • Number of events: An object that reacts to a number of synchronous and asynchronous

calls.

• Number of synchronous calls: The total number of synchronous calls from one object to other objects, either to get or set some data/resource.

• Number of asynchronous calls: The total number of asynchronous calls from an object to other objects.

• Depth of Finite State Machine (FSM): Measures the depth of an indirection where an ob- ject’s behaviour and the states of this behaviour are described by FSM.

• Number of data classes: The total number of data classes an object uses or refers to. • Number of extended state variables: The number of variables an object’s FSM needs to

2.5. EVALUATING SOFTWARE ARCHITECTURE

deal with the machine’s synchronisation aspects.

• Depth of inheritance tree: An object’s total depth from the base class in the system’s in- heritance tree.

The source of faults is also predicted by other metrics. For example, below are two sets of some metrics that were shown to be useful indicators of fault-prone modules in the telecommu- nications domain (Khoshgoftaar -1996), which have been discussed inClements et al.[2002a].

Call graph metrics

• Modules used: This module uses a number of modules, directly or indirectly, including itself.

• Total calls to other modules: This is the number of calls to entry points in other modules. • Unique calls to other modules: This module calls a number of unique entry points in other

modules.

Control-flow graph metrics

• If-then conditional arcs: This is the number of arcs that are not loops and contain a predicate of a control structure.

• Loops: This is the number of arcs that contain a predicate of a loop construct.

• Span of conditional arcs: Within the span of conditional arcs, a total number of arcs is located.

• Span of loops: The number of arcs plus the number of vertices within loop control structure spans.

More of the SA metrics can be found inBass et al. [2013],Kan[2003], Jaquith[2007], Ejiogu[1991],Ejiogu[2005], andCarroll et al.[2007].

2.5.2.2.2 Simulations, prototypes and experiments: Creating and clarifying the architec-

ture may be achieved by building a prototype or a simulation of the system, but creating a de- tailed one for review purposes is usually expensive. A portion of a normal development process typically contains these artefacts; therefore using these artefacts during a review or in answering questions encountered during the review, becomes a typical and natural process.

The answer to an issues raised by a questioning technique, could be solved by using sim- ulation or prototype. Thus, if the review team asks “Can you support this assertion with any evidence?”, one convincing answer would bethe result of a simulation. This questioning tech- nique is useful in the pattern case. For example, What evidence do we have that provesPooling patterns support performance?

2.5.2.2.3 Automated tools and architecture description languages: Representing archi-

tecture through the use of formal/semi-formal notations and languages has been common during the last two decades. One of the most popular utilised architectural semi-formal notations and is

the Unified Modelling Language (UML),Merilinna et al.[2004],Milicev[2009], andSilingas et al.[2011].

Emerging from the academic and industrial research communities are several architecture description languages, such as ADLs (discussed earlier in Chapter 2). Creating, maintaining, and analysing architectures are dependant upon the language and tools maturity. If ADL can describe the behaviour of the system then it most certainly has the tool environment to generate a simulation of the system. Simulations like this provide early insight into architecture inaccu- racies that lead to behavioural errors and reveal performance bottlenecks. Also, many of these tool environments can turn architectural specifications into executable source code by using the right tools, if available.

2.5.2.2.4 Software Performance Engineering (SPE): To see whether a system as designed

will meet its performance constraints, architecture could be examined through the use of a hybrid analysis technique called SPE. SPE is not constrained in a Yes or No result. It is so much more than that. Its purpose is to help the architects and/or designer illuminate and navigate among the trade-offs that are available to them. The intent is to build in performance rather than add it to the system’s architecture / design by scheduling SPE evaluations early and often.

Table 2.9: Comparisons between different evaluation techniques, afterClements et al.2002a.

Quality Attribute(s)

Technique covered Approach(es) Used When applied

Questioning Techniques Questionnaires

and checklists

Various Predefined domain spe-

cific questions

Can be used to prompt architect to take certain design approaches, or any time thereafter. Scenario-based

methods

Various; either non- run-time attributes such

as modifiability or

run-time attributes such as security.

System-specific sce- narios to articulate specific quality at- tribute requirements; scenario walk-through to establish system’s response.

When architecture de- sign is complete enough to allow scenario walk- through.

SAAM Modifiability, function-

ality

ARID Suitability of design

SNA Security

Measuring Techniques

Metrics Various; often empha-

sise modifiability and reliability.

Static analysis of struc- ture.

After architecture has been designed.

2.5. EVALUATING SOFTWARE ARCHITECTURE

Simulations, prototypes, experiments

Various; often empha- sise performance, func- tionality, usability.

Measurement of the ex- ecution of an artefact.

After architecture has been designed.

RMA Performance oriented to

real-time systems.

Quantitative static anal- ysis.

After the process model

has been built and

process-to-processor allocations have been done.

ADLs Various; tend to concen-

trate on behaviour and performance. Simulation, symbolic execution. When architectural specifications are complete. Hybrid Techniques

SPE Performance. Scenarios and quantita-

tive static analysis.

When performance con- straints have been as- signed to architectural components.

ATAM Not oriented to any

particular quality at- tributes, but historically emphasises modifiabil- ity, security, reliability, and performance.

Utility trees and brain- stormed scenarios to articulate quality at- tribute requirements; analysis of architectural approaches to identify sensitivities, trade-off points, and risks.

After the architecture design approaches have been chosen.

Different common Software Architecture Evaluation (SAE) approaches have been briefly discussed, each with its own capabilities. It is also explained that each evaluation method has its own use. Table 2.9 illustrates the comparisons between different SAE approaches.

Most of the above evaluation methods mentioned above can contribute in building analyt- ical models, which could involve in the new software model-driven development approaches, automation checks with formal scenarios, and measuring techniques, in order to explore the qualities within the architecture or software patterns.

In the next section, some specific evaluation methods that contribute to the SAE in particular will be discussed.