Results - Empirical studies on software architecture

4.2 Empirical studies on software architecture

5.1.2 Results

RQ1: How important are NFRs to practitioners?

The first questions where about the role of NFRs in their habitual software development practices:

• 96.0% of respondents consider NFRs in their development practices. • 73.0% said that NFRs are at the same level as functional requirements. • 57.0% said that NFRs are used to make architectural decisions. Respondents rated (in a Likert scale) nine types of NFRs which comprises five of the six quality characteristics of the ISO/IEC 9126 [117] (functional- ity was not included), and the other four were suggested by the testers of the survey: reusability and three non-technical types (standard compliance, cost, and organizational). The respondents rated these types of NFRs with respect to the importance that they have in their projects (see Figure 5.2). In this figure we can see that reliability is the only type of NFR considered

5.1. FIRST EMPIRICAL STUDY

critical by a majority of respondents. While organizational and portability types of NFRs have a median of medium importance, the rest of NFRs have a median of important importance. It was a bit surprising for us that the cost requirements and organizational requirements were of lower importance (in comparison to others). This situation may be dependent on the participants role in the software development (e.g., business role vs. technical role), but if we only consider the participants with project management role the results indicate that they are more worried about reliability with a median of critical importance.

Answer: NFRs are clearly important for practitioners, even if only 73.0% of respondents considered them as important as functional requirements, and, more surprisingly, only 57.0% used them to make architectural decisions (here is worth to remember that not all the survey participants were architects). In fact, almost all the types of NFRs analyzed were considered important by the respondents (i.e, its median is important ).

RQ2: Is the NFRs importance dependent on the architectural style used?

The same analysis was repeated, but in this case limiting the considered answers (A) to the respondents that used NFRs to make architectural decisions, and these were grouped in the three architectural styles studied2:

• 3-Layers architecture (A = 26 answers). The 3-Layers architecture shows that all types of NFRs with the exception of portability, have a median of important importance.

• Service-Oriented Architecture (SOA) (A = 20 answers). In the case of MVC, the reliability has a median of critical importance, and only portability has a median lower than important importance.

• Model-View-Controller (MVC) architecture (A = 24 answers). In the case of MVC, reliability has a median of critical importance.

2_{Note that some respondents answered to develop using more than one type of archi-}

It is worth to mention that limiting the answers in this way, all the respondents considered that at least the importance of any requirement is of marginal importance, having no respondent saying that a requirement is not important at all. A general observation is that the respondents that use NFRs to make architectural decisions gave higher rates to all the types of NFRs, which is an indicator that they are more concerned with NFRs.

Answer: it seems that the importance of NFRs is not hardly condi- tioned by the use of a particular architectural style, but the respondents that work with these architectural styles are more concerned with NFRs in general.

RQ3: Are NFRs well integrated in the current development tools? In this survey we asked about the use of tools in software development, the first question was about a tool to help in the analysis of NFRs compliance. We obtained that 80.0% of respondents declared that the development tools that they use are not well-suited for analyzing the compliance of the specified NFRs, whilst 70.0% would like to have them.

Answer: NFRs are not well integrated in the current development tools, and practitioners would like to have support for them.

RQ4: What is the desired level of interaction with semi-automatic tools?

We also asked questions about the amount of interaction desired with an hy- pothetical support tool for implementation and design tasks. We considered the following tasks:

A. Generation of the skeleton code.

B. Generation of the code for a specific technology. C. Selection of the architectural style using NFRs. D. Selection of the technological style using NFRs.

5.1. FIRST EMPIRICAL STUDY

Architectural style and technological style were defined in the survey as follows:

Architectural style We understand architectural style as the collection of the main elements that compose the software system and the strategy of communication used between them. Examples of software architec- tures are: 3-layered architecture, service oriented architecture, client- server, etc. A software system can be designed as a composition of many architectural styles depending on its needs.

Technological style A technological style is a set of technologies to con- struct the elements that compose the software system. A technological style must consider all necessary technological roles of the implementation: platform, programming languages, libraries, technological stan- dards and external services (e.g., database management systems or authentication services). The technologies that takes part in a technological style must be able to work jointly.

The respondents had to choose one of the following interaction levels for each task:

1. I would not use any support tool to perform this task.

2. The support tool should ask me before taking any decision.

3. The support tool should ask me only before taking the relevant decisions.

4. The support tool should take the decisions for me but later I would check them.

5. The support tool would take the decisions for me without further con- firmation.

The results obtained in these questions are presented in Table 5.3 (rows represent analyzed tasks and columns the interaction level).

Table 5.3: Desired interaction levels 1 2 3 4 5 A. 8.3% 15.0% 23.3% 41.7% 11.7% B. 8.3% 20.0% 26.7% 41.7% 3.3% C. 13.8% 13.8% 50.0% 20.7% 1.7% D. 12.1% 20.7% 39.7% 25.9% 1.7%

Answer: in general practitioners want to have the last word in the architectural decisions made, this means that a completely automatic tool won’t suit their needs. It is also evident that practitioners are more inclined to trust a tool for implementation tasks rather than design tasks.

RQ5: What is the current state of adoption of MDD techniques by the industry?

Finally, we asked about the use of MDD tools, a great part of the respondents (78.3%) said that they do not use MDD in their daily projects, but 51.7% of them declare to know what is MDD. Eclipse (with a 35.0%) seems to be the most known platform for MDD, and MDA (with 48.3%) the most known initiative.

Answer: MDD techniques are far from being adopted by the industry. Even if the techniques are known, practitioners are still reluctant to use them.

5.1.3 Discussion

After analyzing the previous results we had some observations.

• One observation is that, surprisingly, only the 57.0% said to use NFRs to make architectural decisions, this result does not support what is said in the literature about the importance of NFRs in architecture (see Section 1.1.2).

5.1. FIRST EMPIRICAL STUDY

• Another striking observation is that while 80.0% are not able to check their NFRs, the 70.0% would like to have such kind of tool. For us this is a clear indicator that there is an unsatisfied need in software industry.

• About the use of tools, we found an indicator that architects want to have the last word in decision-making. This finding has driven some of the design decisions of the method presented in Section 7.2.

After performing this survey, we decided that it was necessary to obtain more empirical evidence of the current state of practice, in particular for the software architecture area.

0 5 10 15 20 25 30 35