Round 2

Questionnaire Design

We checked each item of round one critically, whether or not it was appropriate for answering our RQs and being queried in the next round. Thus, items of round 1 were consolidated or excluded. Subsequent, we reduced the contextual information given by the expert and merged related items, in order to find the core of the stated issues.

In the end, we identified 34 items as relevant for assessing them in round 2. Based on those items, we created the questionnaire for the second round. The resulting questionnaire assessed 34 items related to the following topics:

89 A Framework for Modeling and Improving Agile Requirements Engineering

• stakeholder and user involvement (6 items) • understanding agile and agile values (6 items) • RE methods (10 items)

• iteration planning and estimation (6 items) • format of requirements (6 items)

The items were grouped into sections by the aforementioned topics. After each section, the experts had the opportunity to give additional comments in free text form. They rated each item using 7-point Likert items (see example in Figure 27). Moreover, they could choose giving no statement.

Figure 27: Exemplary item of round 2

Data Analysis

To sum up, we received responses from 23 experts. For each item we calculated mean, variance and standard deviation. Additionally, we created a diagram showing the distribution of experts’ opinion (see example in Figure 27) and discussed on the meaning of findings.

Results

Calculation of mean, variance, standard deviation and the diagram of experts’ opinion can be found in Appendix IV. The result report of round two can be found in (Schön et al., 2017f).

Stakeholder and User Involvement. Experts agree that it is necessary to have a direct

involvement of user and stakeholder so that product development will succeed and user´s needs can be fulfilled (Item 1.1, Item 1.2). In this context, there is a high correlation to both the type of product and the reachability of the target group (see comment expert 9). In addition, lightweight methods need to be applied in order to involve users and stakeholder (see comment expert 15). Furthermore, the continuous communication is an important aspect in this context (Item 1.6). The experts agree that face-to-face communication is an appropriate technique not to miss important expectations from different stakeholders (Item 1.3). Moreover, we can state that the experts differ whether the requirements need to be validated with direct end users before development can start (see Item 1.4). This may correlate with the culture of a company in terms of HCD (International Organization for Standardization, 2010). Moreover, the experts have a different opinion concerning the moment of receiving feedback from the stakeholder (Item 1.5).

90 A Framework for Modeling and Improving Agile Requirements Engineering

Understanding Agile and Agile Values. The experts agree that in ASD a lot of decisions are in

responsibility of the development team. Therefore, stakeholders need to understand that they cannot take part in every decision (Item 2.3). However, decisions concerning the change of scope and features of the product should be discussed with stakeholders (see comment expert 23). Moreover, the panel agrees that stakeholder do not comprehend that requirements should be negotiable with the development team (Item 2.1). In terms of this, expert 7 stated that the development team should decide about technical realization, whereas the Product Owner together with the stakeholders should take the decision concerning functional questions. In ASD there is often a kind of misinterpretation and Agile is equalized to chaos of requirements or missing documentation. This leads to resistance due to loss of control and fear of increasing risk (Item 2.5). In terms of this, there is a strong correlation to the phase of agile transition of an organization (see comment expert 24). In general, the panel differs concerning a pre-performed requirement analysis (Item 2.2). However, expert 23 state that it is useful to consolidate business requirements in the beginning, without being too detailed.

Requirements Engineering Methods. The panel strongly agrees that in ASD the continuous

management of requirements is important since not all of them are fixed in the beginning and they may change over the course of time (Item 3.4). Additionally, experts agree upon that established techniques for elicitation and evaluation of requirements are often to slow (Item 3.5). Techniques need to be applied, which support the sharing of knowledge among the whole development team (Item 3.6). Furthermore, we can observe strong agreement among the panel that in ASD requirement documents need to be adapted to changing conditions with low effort (Item 3.10). The experts refuse that in ASD do not exist a technique to define NFRs (Item 3.9). This may lead to the assumption that there is no longer a problem concerning the handling of NFRs in ASD. Expert 16 recommends the following best practices for handling NFRs: Definition of Done, smoke test, performance test and penetration test. Moreover, the experts agree that missing styleguides and prototypes can lead to the problem that developers implement their own assumptions of UX, which can result in a bad UX for users (Item 3.2). In summary, we can conclude that the established understanding of RE and known techniques need to be adapted to the context of ASD.

Iteration Planning and Estimation. Experts agree that it is important to focus on the refinement

of requirements for the short-term iterations in order to react flexible on new information (Item 4.1). Moreover, an outlook on the upcoming iterations is important for coordinating related projects. Nevertheless, this outlook should not be a binding one (Item 4.3). The panel agrees that requirements should be split in such a manner that they can be implemented in one iteration by adding value to the product (Item 4.5). Furthermore, the sight of the big picture should not be loosed due to the complexity of requirements (Item 4.6). The results have impact on the modeling of agile RE process models and how requirements are refined. Figure 28 displays the degree of refinement of requirements over the course of time.

91 A Framework for Modeling and Improving Agile Requirements Engineering

Figure 28: Degree of refinement of requirements

Format of Requirements. The panel agrees that requirements must be formulated as objectives that describe the problem area so that the creativity in solution finding is not restricted (Item 5.3). In addition, the benefits of a requirement must be justified in order to make added value of the implementation clear as well as decisions for a specific requirement comprehensible (Item 5.6). Experts agree that requirements must be captured in such a way that detailed test cases can be derived from them for quality assurance (Item 5.4). Requirements must be formulate in a clear manner in order to avoid uncertainty in terms of implementation (Item 5.5). Moreover, the experts agree that in ASD functional or technical dependencies lead to a high coordination effort (Item 5.1). The additional comments stated by the experts reveal that the items related to the category format of requirements are also applicable in project using non-agile approaches (see comment expert 11 and expert 12).