In conducting the research reported in this thesis, an initial motivation was present already in the form of the following three interrelated statements:
Firstly, “ If we are to agree on what it means to document a software architecture, we should establish a common basis for what it is we’re documenting”,Bachmann et al.[2011, p3]. Sec- ondly, “Architecture assessments are essential for avoiding, identifying, or mitigating risks”, Mistrı́k et al.[2014, p11]. Thirdly, “Using a pattern or style means making successive design decisions that eventually result in an architecture”,Bachmann et al.[2011, p35].
The foundational theory behind this work (as expressed by many others includingQin et al. [2008],Bachmann et al.[2011], andBass et al.[2013]) is that SA, SAE, and SPs, are strongly related. Whilst the theory might hold, the various artefacts and practices that have been devel- oped over time (including their documentation processes) do not, which prevents the maximum utilisation of many of the existing achievements within SPs, SA, and SAE domains.
This (current) situation has led to an initial aim of finding ways to realize current issues regarding SA descriptions, SAE methods, and documentation of software patterns/styles1, in order to identify areas of improvement.
Preliminary research (reported in Chapter 2) shows that SA and software Styles/Patterns have a strong relationship and improving one will improve the other. It also explains that SAE is affected by several factors, such as SA description languages including SPs, the level of for- mality, documentation techniques, standardisation, and selected evaluation methods. After all: “Architectural evaluation of a software/system is crucial for its success”,Reussner et al.[2005]. In order to assess the architectural fitness of software systems, a number of evaluation meth- ods have been proposed, such as Architecture Tradeoff Analysis Method (ATAM) and Software
1
Different books distinguish between styles and patterns, where styles are considered for architecture and patterns are considered for design. In this thesis, the difference between style and pattern is not critical, both are considered as a recurring solution to the same problem in different contexts.
1.2. INITIAL MOTIVATION AND RESEARCH AIM
Architecture Analysis Method (SAAM), and Active Design Reviews (ADR). Most of the exist- ing methods are intended for evaluation of a single architecture at a certain point in time.
Furthermore, the results from using MANY/SOME methods are highly dependent on the person performing the evaluation and generally cannot be used to compare different architec- tures. Most of the current mature architecture evaluation methods, such as ATAM and SAAM use qualitative techniques that are typically applied through the use of scenarios. Consequently, the interpreted results depend heavily on the choice of scenarios to evaluate certain QA. The gen- eration of these scenarios is solely based on the vision and requirements of the stakeholder(s). The conflicts between stakeholders’ requirements raises a major challenge to software architects. This was noticed by Avison et al. [1999] and Baskerville et al. [2004] during their studies for ATAM and SAAM methods. The same challenge is confirmed byQin et al.[2008, p227-228], Bass et al.[2013, p401], andShreelekhya et al.[2016].
Moreover, SAAM has at least one pitfall in that it does not provide clear quality metrics for the architectural characteristics that are being analysed,Clements et al.[2002a]. In addition, the assessment of some properties requires experts. For example, evaluation of security is not usually easy, due to the lack of adequately qualified resources. Another major problem is that most of the evaluation methods are ad-hoc processes or minimally-automated and thus they are prone to error. Additionally, most of these methods, especially the semi-automated ones, are developed to evaluate one quality attribute, such as in the work done byMoreno et al.[2008] to analyse performance.
Further research, as reported in Chapter 2, shows that the SA qualitative evaluation methods received much more attention than quantitative evaluation methods, which renders the latter methods as more limited and in need of more research, and exhibiting less maturity.
In concluding this preliminary research, the necessity to go further in order to investigate SPs documentation and utilization, SA, and SAE methods, it was important to understand and discover the factors influencing SAE approaches.
Hence, based upon the preliminary study, questions have been formulated (as below) so as to promote understanding of the SA description and evaluation, and to develop a viable solution: 1. What effect do description languages, standardisation, evaluation methods, modelling tech-
niques and their documentation have on ‘SPs and SA’ utilisation and evaluation?
2. How can ‘SPs and SA’ utilisation and evaluation be improved, including minimising the effects of any hindrances?
Answering these questions is important, since “An unsuitable architecture will precipitate disaster on a project”,Clements et al.[2002a].
Thus, the aim of this research is to investigate the SA and SAE domains through liter- ature review, questionnaires, field study, and analyses. As a result, issues and/or links between different disciplines, such as SPs, modelling languages, and SA description and evaluation techniques, were presented, in order to resolve identified issues, bridge the gaps, and overcome the limitations of SAE current methods.
1.2.1
Contributing Factors to the Problem Domain of this Thesis
The following points list the main factors that make this research worthwhile:
1. Lack of architectural evaluation approaches that apply standard languages and that are fully automated.
2. The absence of well-known, wide-spread standardized languages and tools in current ar- chitectural evaluation studies.
3. There are no/few robust architectural evaluation models that can help architects and de- velopers to check their architecture against intended and required ‘qualities’ with no need for experts.
4. There is no existence of a facility (such as a database, repository, or thesaurus etc.) that demonstrates the relationships between SPs and QAs based on reliable references. 5. There seems to be a dearth of experts on evaluation methods at an architectural level.