Model checking and Testing

In this section, we introduce works that discuss the use of model checking to generate test cases to test real implementations. We start with works that use model checking in latter phases of the life-cycle and, finally, we discuss the use of model checking when the model is not vulnerable.

Model Checking and Security Testing

Recently, model checking has been used to verify the security of already deployed security protocols. For example, Armando et al. [ACC+08] discov- ered a severe security flaw in the SAML-based Single Sign-On for Google Apps [Goo08]. The authors fed SATMC with (i) a transition system for the behavior of the participants and the attacker, (ii) a set of LTL constraints for modeling properties on the communication channels, and (iii) a LTL formula modeling the non-injective agreement property [Low97]. SATMC returned an attack in which a malicious SAML service provider can impersonate a legitimate user at any other service provider within the same federated en- vironment. It is important to point out that the attack has been manually interpreted and reproduced against the implementation.

Guangdong et al. [GGJ+_{13] presented AUTHSCAN, a tool that com-} bines a number of different techniques: model inference, static analysis of client-side script, model checking, and security testing. The first two tech- niques aim at inferring a model from a protocol implementation and will be detailed in Section 2.4. After the model inference step, the model is verified

2.2. DESIGN VERIFICATION ₂₅

by the model checker in the classical way. If the model checker returns a counterexample, AUTHSCAN translates it into a concrete test case. The translation replaces the abstract values of the counterexample with the real values learned during the model inference step. Furthermore, AUTHSCAN uses a user-defined test oracle to produce the test verdict after the test ex- ecution. The authors evaluated their tool against eight implementations of BroweserID (now called Persona [The13b]), Facebook Connect [Fac13], Windows Live ID [Mic13], and custom user authentication protocols. The authors reported that AUTHSCAN discovered seven security vulnerabilities of different type e.g. replay attacks, CSRF attacks, secret token leaks, and guessable tokens.

Model Checking and Mutation-based Security Testing

If the model checker does not find a counterexample in the model, it does not imply that the implementations are secure as well. In fact, implementations may be still vulnerable due to errors introduced by the developers. Dadeau et al. [DHK11] and later Büchler et al. [BOP11, BOP12a, BOP12b] proposed to apply the mutation-based testing technique to detect vulnerabilities.

Dadeau et al. proposed to mutate the model of a security protocol by injecting faults. Faults are injected by using the so called mutation oper- ators. If the model checker finds an attack, the attack is used as a test case. Büchler et al. [BOP11, BOP12a, BOP12b] went beyond the prelimary work of Dadeau by proposing SPaCiTE, a tool for mutation-based, semi- automatic, security testing of web applications. The tool works as follows. First, the user selects the vulnerability to inject into the model. Then, the model checker verifies the security property on the mutation. If the model checker finds a counterexample, then it is concretized and executed against the real implementation. The concretization is done in two steps. First, the counterexample is mapped into concrete browser actions. To simplify the mapping, the authors proposed an intermediate language called WAAL (Web Application Abstract Language). Then, the counterexample is inter- preted and executed against the web application. SPaCiTE has been assessed

against two lessons of WebGoat [The07a], an application vulnerable on pur- pose for educational purposes. The authors reported that they manage to detect successfully stored XSS, and lack of authorization vulnerabilities.

2.2.3 Discussion

In this section, we have presented works in the area of design verification via model checking. Model checking was applied to verify the design of security protocols, web services, and business processes, discovering previously un- known design flaws both during the design and at the deployment phases. Moreover, model checking has been proposed as a tool for supporting testing and security testing of real implementations.

However, these works showed the following shortcomings. First, the de- sign verification via model checking focuses on the automatic detection of flaws in a model of the system under verification and falls short on testing the real system. In fact, the counterexamples returned by model checkers prove only that the model is flawed and it does not say whether the real system is also vulnerable. For example, a real system may solve the security flaw with additional and undocumented behaviors. As a result, in order to detect the flaw in implementations, the counterexamples are interpreted and reproduced against each implementation. To date, this activity is still done manually. Second, the mutation testing techniques have been assessed only on small applications to detect known vulnerabilities and there is still a lack of evidence of the scalability of these approaches to real systems. More- over, the translations between models and real systems are specific to the web application domain, and they do not support cryptographic primitives, message composition and parsing.

In this section, we also presented AUTHSCAN whose authors claim it is able to automatically execute counterexamples against real implementations. It must be pointed out that the contribution of this thesis relatively to the execution of counterexamples is anterior to AUTHSCAN. Furthermore, the authors did not provide sufficient details about their techniques nor provided the tool. As a result, we could not perform any comparative analysis.