Verification and Test Campaign

A verification campaign is a multiple execution of the verification workflow. This solves the practical problem of verifying several models. Similarly, the test campaign consists of the executions of several test cases.

Figure 7.5 shows the editor for the test campaign manager. On the left- hand side, the editor displays the available models. Models are shown in a tree-like form. On the right-hand side, the editor shows the list of test cases generated and the IUTs available. The user selects the test cases and the IUTs, and she runs the campaign. At the end of the execution, the tool displays the HTTP conversations for off-line analysis. The result of a campaign is organized into tables. In addition, the tool logs the results and HTTP messages of all the test for future inspections.

7.3. CONCLUSIONS ₁₆₁

The result of a verification and/or test campaign is organized into tables together with the result and/or test verdict as well as the labels (if any).

7.3

Conclusions

In this chapter we showed how some of the techniques presented in this dis- sertation have been transferred to SAP. We described the security analysis of the SAP implementation of SAML SSO, supporting developers in taking design and implementation decisions. In addition, we presented the design of a tool that eases the security analysis of protocol design, the assessment of protocol configuration, and the analysis of protocol deviation. In addi- tion, the tool enables to test real implementations using counterexamples as abstract test cases.

Chapter 8

Conclusions and Future Work

In this chapter we summarize the contribution of this thesis with respect to the objectives that we have set in Section 1.1. Then, we give an overview of possible future work that could be carried out based on the results presented in this thesis.

8.1

Contributions

State-of-the-art security testing technologies do not provide automated sup- port to the discovery of logic vulnerabilities in multi-party business applica- tions. In this thesis, we have addressed the shortcomings of these technolo- gies in order to support the automated detection of logic flaws.

We started in Chapter 4 with the design verification via model checking of the SAML SSO and OpenID authentication protocols. Starting from the specifications written in natural language, we wrote formal models captur- ing the behavior of the protocol participants, message structure, and com- position of participants. We showed that when formal models are available, model checking can automatically discover flaws into the logic of the protocol design. However, the discoveries are not directly applicable to the real im- plementations. Moreover, we showed that there is still a substantial amount of manual work required to confirm the presence of the flaw in real imple- mentations. Finally, we discovered that the design flaw can be exploited as a launching pad of XSS attacks in the SAML-base SSO for Google Apps.

All our findings have been discussed with members of the OASIS Security Services Technical Committee and a SAML V2.0 Errata has been redacted and approved [OAS12].

In Chapter 5 we tackled the first objective of this thesis that is testing real implementations starting from the attacks returned by a model checker. We proposed an approach that fills the gap between formal model and real implementations by the means of model instrumentation. The model instru- mentation calculates a set of program fragments that encode the message generation, message parsing, and the check of the incoming messages against the current state of the participants. The fragments are then executed in the order established by the counterexample.

The approach of Chapter 4 and Chapter 5 is applicable when the speci- fications are available. In Chapter 6, we proposed an automated black-box approach that does not require a model as input. Our approach infers a model from a set of network traces. Afterwards, the model is used to gen- erate test cases following a number of attack patterns. Finally, tests are executed against the real implementation and an oracle decides whether a property of the application has been violated.

This thesis has been carried out in an industrial context. This allowed us to balance the design of testing techniques with their pragmatical application to real world applications. The techniques of Chapter 4 and Chapter 5 have been implemented in an industrial tool, while the black-box testing technique of Chapter 6 is implemented as a proof-of-concept. The former tool has been used to test three implementations of SAML SSO and two of OpenID detecting the logic flaws discovered by the model checker. Moreover, this tool has been used to support SAP engineers to evaluate the security of the design of their SAML SSO implementation. Furthermore, it is currently used to test the SAP implementation of OAuth 2.0. The second tool implements the black-box testing approach described in Chapter 6. The tool has been used to test 12 eCommerce web application deployments discovering ten previously unknown critical vulnerabilities and about 900 presentations bugs. All the critical vulnerabilities that our techniques discovered have been responsibly disclosed.