presented in the papers.
Research Method Setting Research Question Paper Qualitative Interviews
Interviewing 8 informants (IT managers and ar- chitects) from 6 organizations in the Norwe- gian electricity industry
RQ1 Paper 1
Survey An inter-rater agreement study on 37 IT experts RQ1 Paper 5
Literature Review
Analyzing 5 tools and research prototypes RQ1 Paper 2
Design Science
Developing and testing an information extrac- tion framework
RQ2 Paper 3 and 4
Case Study Using 3 architectural documents from a
telecommunication company to test and eval- uate the framework
RQ2, RQ3 Paper 4
Experiment 1 Evaluating the framework with 19 IT students RQ3 Paper 3
Experiment 2 Evaluating the framework with 21 expert archi-
tects from 5 Norwegian organizations
RQ3 Paper 4
Table 3.1: Summary of research methods
3.2
Research Evolution
This PhD was a journey that traversed more topics than those presented in this thesis. The initial research topic was Improved Management of Software Evolution for Smart Grid Applications. In the inception phase, we made some preliminary observations in the electricity industry and in the literature. Two conceptual papers were published reporting the conducted literature reviews. One paper is still slightly connected to this thesis and is presented as a supporting paper (Paper 0) while the other one is not included in the thesis [ACC12]. The research topic after 1.5 years shifted from the focus on software evolution in Smart Grid to architectural decision making for large-scale software systems. Smart Grid as an instance of large-scale systems is still connected to the research, but the research also concerns other large-scale systems such as those in telecommunication or banking area.
Figure 3.1 shows how this PhD research has evolved over the time (it excludes the first 1.5 years of the research where the topic was different). It demonstrates different phases of the research and different studies that have been conducted in each phase and the papers that has been delivered during the research. After conducting several qualitative inter- views, the more specific interest of the research became reusing architectural decisions
Figure 3.1: The research evolution and phases
and enhancing architectural decisions guidance from previous decisions. A literature re- view analyzed the current tools that support reusing architectural decisions and enhancing decision guidance. Completing the inception phase, accelerating the decision guidance enhancement became the main focus of the research. Several methods were employed to develop and evaluate a framework supporting automated decision guidance enhancement. The journey ended by conducting a survey to initiate an inception phase for the future study. It should be noted that the path shown in Figure 3.1 is a retrospective view on the journey; the PhD roadmap was not designed prospectively in this manner. It was evolved over time and the final roadmap is presented here.
CHAPTER
4
Results and Analysis
This chapter summarizes the results of this thesis. The subsections are written based on the results of studies which were introduced in Chapter 3. Each section gives a summary and contribution of the results of each step of the research. The details of the results are available in the selected papers presented in Part 2 of the thesis.
4.1
Architectural Decision-Making in Enterprises: State-
of-Practice
The first step of this PhD research was to explore the large scale enterprises to find out the main processes and issues on making and reusing architectural decisions by consid- ering the relationships among the enterprises and other actors of the ecosystem. This exploratory study partially answered the first research question of this thesis which is presented in Section 3.1. The full results of this study is published in Paper 1. In the following, the summary of the results are presented.
4.1.1
Architectural Decision-Making Approaches
The results of our study in line with the literature [vHA11], show that most of the large scale enterprises are not using systematic approaches such as the architectural trade-off
analysis method (ATAM) [KKB+98] to make and evaluate their architectural decisions. Nevertheless, it does not imply that the organizations are making their architectural deci- sions totally unsystematically. Both the results of our study and findings from literature, show that the enterprises first identify architecturally significant requirements (architec- tural analysis), then find different candidate solutions for the requirements (architectural synthesis), and finally validate the chosen solution against the requirements (architectural evaluation) [vHA11]. In spite of similarities, different companies of our study have vari- ous procedures for each of the mentioned processes. For instance, for architectural eval- uation, some apply proof-of-concept, while some launch industrial prototype to evaluate the chosen solution.
4.1.2
Effect of Software Ecosystem Relationships on the Architec-
tural Decisions
The case of our study was the software ecosystem (SECO) of the Norwegian electricity industry. Figure 4.1 visualizes the software supply network in this SECO.
Figure 4.1: Current software supply network in the Norwegian electricity industry The results of our study show that the relationships among the actors of a SECO could significantly affect the architectural-decision making process for each of the actors. In this study, as Figure 4.1 shows, these effects are : 1) effect of regulators on the architectural decisions of customers, 2) effect of vendors on the architectural decisions of customers, and 3) effect of customers on the architectural decisions of vendors.