This section starts by explaining how the goals defined in the problem state-ment in Section1.2are attained, i.e. how the problem statement is solved. It further explains whether the research goals defined in Section1.3are attained.
Section 6.3.1explains the former, while Section 6.3.2explains the latter. The problem statement points and the research goal points are referred to by their identifiers, (e.g. Problem statement point 1, and RG.01). The reader is therefore encouraged to look at Section 1.2 for the problem statement, and Section1.3 for the research goals in order to understand the points that are being referred to.
6.3.1 Problem statement goals
• Problem statement point 1: In Chapter 2, the author performed a survey of state-of-the-art security testing methodologies, with a special focus on Web applications. This was done by gathering and analyzing relevant information from academic books, scientific papers and Web sites.
With this, problem statement point 1 is fulfilled.
• Problem statement point 2: The author defined a set of criterions in Section 2.7. Using those criterions as basis, the author elicited Agile Security Testing as the most adequate security testing methodology for Web applications. However, for reasons given in Section2.7, the author’s selection criteria are to some extent subjective. In turn, this could have an affect on the resulting decision.
Initially, Agile Security Testing did only fulfill point 2a in the problem statement, and did not fulfill point 2b. Neither did it fulfill criterion C.05 and criterion C.06 defined in Section 2.7. Therefore, the author provided a solution for point 2b, criterion C.05 and criterion C.06 as shown in Section 4.1. The resulting solution was named Extended Agile Security Testing (EAST). With this, problem statement point 2 is fulfilled.
• Problem statement point 3: In Chapter 3, the author presentet the current situation of the AIS group at CERN, and described the SDLC that is being used by the AIS group (Scrum). Furthermore, an explanation to why there is a need for a security testing methodology in the AIS group was given. This was done by:
1. Describing the security testing methodology applied by the AIS group, and giving an explanation to why the current security testing method-ology is not sufficient.
2. Conducting a risk analysis of the AIS group’s software systems, by using the most important assets that are handled by their software systems as a starting point.
In Section 4.2 the author described in which phase of Scrum the various EAST steps are integrated. Finally, in Section 5.1the author completed the integration by describing how, why, and by whom the EAST steps are carried out. With this, problem statement point 3 is fulfilled.
• Problem statement point 4:
1. In Section 2.4, 2.5 and 2.6, different security testing tools were de-scribed and evaluated, in which Acunetix WVS was selected. Fur-thermore, in Section 2.5.1, the author justified why Acunetix WVS was selected. With this, problem statement point 4a is fulfilled.
2. In Section 4.3the author explained which security tests were to be performed, and how the security testing process would be conducted.
Furthermore, in Section 5.2the author conducted the security test-ing of DAI and MAG. The security tests were carried out by three testers. As mentioned in Section 6.2, the number of testers was a side effect of the limited time and resources. Nevertheless, a secu-rity test containing four iterations (two iterations using the current methodology, and two iterations using the EAST methodology) was carried out. With this, problem statement point 4b is fulfilled.
3. Based on the test results given in Section 5.3, an evaluation of the security testing methodologies has been made in Section 6.1. With this, problem statement point 4c is fulfilled.
6.3.2 Research goals
This section describes whether the author has attained the research goals de-fined in Section 1.3. This is done by associating the problem statement with the research goals, and thereby describing how the research goals are attained
RG.01
RG.02
RG.03
RG.04 Problem statement fulfilled
1 2 3 4a 4b 4c
Research goal attained
Figure 6.2: The left hand side of the figure shows the problem statements that are fulfilled. By fulfilling the problem statements, the research goals are at-tained, as shown on the right hand side of the figure.
as a result of fulfilling the problem statement (the fulfillment of the problem statement goals are described in Section6.3.1). Figure 6.2 illustrates what is explained in the points below.
• RG.01: By fulfilling problem statement point 1, 2, 3 and 4, the author gained knowledge of state-of-the-art security testing methodologies, with a special focus on Web applications. With this, the author attained research goal RG.01.
• RG.02: By fulfilling problem statement point 1 and 2, the author found and evaluated security testing methodologies for Web applications. With this, the author attained research goal RG.02.
• RG.03: By fulfilling problem statement point 3, 4b and 4c, the author implemented one security testing methodology (at a proof of concept level) for Web applications into the SDLC applied by the AIS group at CERN, and evaluated it. With this, the author attained research goal RG.03.
• RG.04: By fulfilling problem statement point 4a, the author got an overview of the different security testing tool categories, along with some tool examples for each category (both freeware and commercial). With this, the author attained research goal RG.04.