WordPress plugins

To answer the first and last questions, and to find previously-unknown (zero-day) vulnera- bilities, we run WAPe with a set of 115 WordPress (WP) plugins (WordPress, 2015), 5 of which with vulnerabilities registered in CVE (CVE,2015).

WordPress is the most adopted CMS and supports plugins developed by many different teams. We selected 115 plugins from different tags (arts, food, health, shopping, travel, authentication, popular plugins and others) and distributed by several ranges of downloads, from less than 2000 to more than 500K. The popular plugins fit in this last range, having

4.4 Experimental Evaluation

Plugin Version Real vulnerabilities Total FPP FP

SQLI XSS Files* _{SCD CS HI}

Appointment Booking Calendar** _{1.1.7 1 3 4 1}

Auth0 1.3.6 1 1 Authorizer 2.3.6 2 2 BuddyPress 2.4.0 0 1 Contact formgenerator 2.0.1 11 11 CP Appointment Calendar 1.1.7 2 2 Easy2map** _{1.2.9 1 2 3}

Ecwid Shopping Cart 3.4.6 1 1

Gantry Framework 4.1.6 3 3

Google Maps Travel Route 1.3.1 1 2 3

Lightbox Plus Colorbox 2.7.2 8 8

Payment form for Paypal pro** 1.0.1 2 2

Recipes writer 1.0.4 4 4

ResAds** 1.0.1 2 2

Simple support ticket system** 1.2 18 18

The CartPress eCommerce Shopping Cart 1.4.7 8 17 25

WebKite 2.0.1 1 1

WP EasyCart - eCommerce Shopping Cart 3.2.3 13 6 29 5 2 5 60

WP Marketplace 2.4.1 9 9 1

WP Shop 3.5.3 5 5 1

WP ToolBar Removal Node 1839 1 1

WP ultimate recipe 2.5 0 1

WP Web Scraper 3.5 3 3

Total 55 71 31 5 2 5 169 3 2

*

DT & RFI, LFI vulnerabilities

**

plugins with vulnerabilities registered in CVE-2015-7319, CVE-2015-7320, CVE-2015-7666, CVE-2015-7667, CVE-2015-7668, CVE-2015-7669, CVE-2015-7670

Table 4.7: Vulnerabilities found by new version of WAP in WordPress plugins.

some of them more than 1M downloads. Figure4.4(a) shows the number of downloads of these plugins and Figure4.4(b) the number of web sites that have these plugins active.

WAPe discovered 153 zero-day vulnerabilities and detected 16 known vulnerabilities. Table 4.7 shows the 23 plugins with vulnerabilities, distributed by 8 classes. The wpsqli weapon detected 55 SQLI vulnerabilities, while the other detectors found the remaining 114 vulnerabilities of the XSS, RFI, LFI, DT, HI and CS classes (last 2 are new). For the known 5 vulnerable plugins (appointment-booking-calendar 1.1.7, easy2map 1.2.9, payment-form- for-paypal-pro 1.0.1, resads 1.0.1 and simple-support-ticket-system 1.2), we confirmed the vulnerabilities using the information about them published in BugTraq (BugTraq, 2015). However, for the simple-support-ticket-system 1.2 plugin WAPe detected more 13 SQLI vul- nerabilities than those that were registered.

Page 2 < 20002K – 5K.5K – 10K10K – 50K50K – 100K100K – 500K> 500K 0 5 10 15 20 25 30 35 Analyzed Vulnerable < 100100 – 500500 – 1K1K – 2K2K – 5K5K – 10K> 10K 0 5 10 15 20 25 30 Analyzed Vulnerable (a) Downloads Page 2 < 20002K – 5K.5K – 10K10K – 50K50K – 100K100K – 500K> 500K 0 5 10 15 20 25 30 35 Analyzed Vulnerable < 100100 – 500500 – 1K1K – 2K2K – 5K5K – 10K> 10K 0 5 10 15 20 25 30 Analyzed Vulnerable (b) Active installs

Figure 4.4: Downloads and active installed plugins of 115 analyzed (blue columns) and 23 vulnerable (orange columns) plugins.

The 23 plugins fit in all ranges of downloads, as depicted by the orange columns of Figure

4.4(a). 16 of them have more than 10K downloads, reaching more than 500K downloads. All ranges of active WP installations contain vulnerable plugins, as shown by the orange columns of Figure4.4(b). 12 plugins are used in more than 2000 web sites. The vulnerable Lightbox Plus Colorbox plugin is active in more than 200,000 web sites (the most used plugin), making these web sites vulnerable to XSS attacks.

Figure4.5presents the vulnerabilities detected by class for the 17 web applications and 23 WP plugins. Clearly SQLI and XSS continue to be the most prevalent classes. Moreover, it is possible to observe that WAPe detects correctly the vulnerabilities it was extended to detect. In both analysis it detected HI and CS vulnerabilities, while LDAPI and SF were only detected in the web applications (not plugins).

All these vulnerabilities were reported to the developers of the web applications and WP plugins. Some already confirmed their existence. All were confirmed by us manually.

4.5

Conclusions

The chapter presents the extension of the WAP tool to detect new vulnerabilities. It addresses the difficulty of extending these tools by proposing a modular and extensible version of the WAP tool, equipping it with “weapons” to detect (and correct) vulnerabilities of new classes. The approach involved restructuring WAP to make it modular and the creation of a new

4.5 Conclusions Sheet10

Page 4 PLUGUNS & WEB APPS

Plugins SQLI 72 55 XSS 255 71 Files 55 31 SCD 4 5 LDAP 2 0 SF 1 0 HI 19 2 CS 5 5 WebApps SQLI XSS Files SCD LDAP SF HI CS 0 25 50 75 100 125 150 175 200 225 250 275

Number of vulnerabilities by class

WebApps Plugins

Figure 4.5: Number of vulnerabilities detected by class in the vulnerable web applications and WordPress plugins.

module to generate weapons, i.e., to generate automatically detectors and fixes to detect and remove new classes of vulnerabilities. To predict false positives the precision and accuracy of the data mining process has been improved, adding more symptoms about false positives and instances.

The new version of the tool was evaluated with seven new vulnerability classes using 54 web application packages and 115 WordPress plugins, adding up to more than 8,000 files and 2 million lines of code. The tool discovered respectively 366 and 153 zero-day vulnerabilities, i.e., 519 previously-unknown vulnerabilities. In our experiments our modular and extensible tool has shown a much higher ability to detect new (zero-day) vulnerabilities than the original version.

5

Learning to Detect Vulnerabilities

Programmers often use static analysis tools to search for vulnerabilities automatically in the application source code, then removing them. However, developing these tools requires ex- plicitly coding knowledge about how each vulnerability is detected (Dahse & Holz, 2014;

Fonseca & Vieira, 2014; Jovanovic et al., 2006), which is complex. Moreover, this knowl- edge may be wrong or incomplete, making the tools inaccurate (Dahse & Holz, 2015). For example, if the tools do not understand that a certain function sanitizes inputs, this could lead to a false positive (a warning about an inexistent vulnerability).

This chapter presents a new approach for static analysis, leveraging classification mod- els for sequences of observations that are commonly used in the field of natural language processing (NLP). Currently, NLP tasks such as parts-of-speech tagging or named entity recognition are typically modeled as sequence classification problems, in which a class (e.g., a given morpho-syntactic category) is assigned to each word in a given sentence, according to estimates given by a structured prediction model that takes word order into considera- tion. The model’s parameters (e.g., symbol emission and class transition probabilities, in the case of hidden Markov models) are typically inferred using supervised machine learning techniques, leveraging annotated corpora.

We propose applying the same approach to programming languages. These languages are artifical but they have many characteristics in common with natural languages, such as the existence of words, sentences, a grammar, and syntactic rules. NLP usually employs machine learning to extract rules (knowledge) automatically from a corpus. Then, with this knowledge, other sequences of observations can be processed and classified. NLP has to

take into account the order of the observations, as the meaning of sentences depends on this order. Therefore it involves forms of classification more sophisticated than classification based on standard classifiers (e.g., naive Bayes, decision trees, support vector machines) that simply verify the presence of certain observations, without considering any order and relation between them.

This work is the first to propose an approach in which static analysis tools learn to detect vulnerabilities automatically using machine learning. The approach involves using machine language techniques that take the order of source code instructions into account – sequence models– to allow accurate detection and identification of the vulnerabilities in the code.

We specifically use a hidden Markov model (HMM) (Rabiner,1989) to characterize vul- nerabilities based on a set of source code slices with their code elements (e.g., function calls) annotated as tainted or not, taking into consideration the code that validates, sanitizes, and modifies inputs. The model can then be used as a static analysis tool to discover vulnerabil- ities in source code. A HMM is a Bayesian network composed of nodes representing states and edges representing transitions between states. In a HMM the states are hidden, i.e., are not observed. Given a sequence of observations, the hidden states (one per observation) are discovered following the HMM, taking into account the order of the observations. The HMM can be used to find the sequence of states that best explains the sequence of observations (of code elements, in our case). To detect vulnerabilities we introduce the idea of revealing the discovered hidden states of the code elements that compose the slice. This is interesting be- cause the state of the elements determines if they are tainted, i.e., if the state may have been defined by an input, which may have been provided by an adversary. This allows the tool to interpret the execution of the slice statically, i.e., without actually running it. Notice that transitioning from a state to another requires understanding how the code elements behave in terms of sanitization, validation and modification, or if they affect the data flow somehow. This understanding is performed by the machine learning algorithm we propose.

The chapter also presents the DEKANT tool – hidDEn marKov model diAgNosing vul- nerabiliTies– that implements our approach. DEKANT first extracts slices from the source code, next translates these slices into an intermediate language – intermediate slice language (ISL) – and retrieves their variable map. Then it analyses that representation, with the assis- tance of its variable map, to understand if there are vulnerabilities or not. Finally, the tool outputs the vulnerabilities, identifying them in the source code.

5.1 Overview of the Approach

The chapter is organized as follows: the next section (Section5.1) gives an overview of the approach for detection input validation vulnerabilities in web applications using static analysis based in a sequence model that learns to classify. Then, the Intermediate Slice Language (ISL) is characterized and described in Section 5.2. The ISL is used to translate PHP slices in a tokenized language, more simple to process by a sequence model. Section5.3

presents the model that receives the translated PHP slices to classify them as being vulnerable or not, detecting thus vulnerabilities. In Section5.4the DEKANT tool that implements the model is presented and in Section 5.5 an experimental evaluation is showed. The chapter ends with discussion, including with related work, and conclusions (Sections5.6and5.7).

5.1

Overview of the Approach

The approach has two phases: learning and detection. In the first, an annotated data set is used to acquire knowledge about vulnerabilities. In the second, vulnerabilities are detected using a sequence model, a HMM.

The HMM captures how calls to sanitization functions, validation and string modification affect the data flows between entry points and sensitive sinks. These factors may lead state to change from not tainted to tainted or vice-versa. However, we do not tell the model how to understand these functions, but train it automatically using the annotated data set (see Section5.3).

The two phases are represented in Figure5.1. The learning phase is executed when the corpus is first defined or later modified and is composed of the following sequence of steps: 1. Building the corpus: to build the corpus with a set of source code slices annotated either as vulnerable or non-vulnerable, to characterize code with flaws and code that handles inputs adequately (see Section5.3.1). Duplicates have to be removed.

2. Knowledge extraction: to extract knowledge from the corpus (the parameters of the model) and represent it with probability matrices (see Section5.3.2).

3. Training HMM: to train the HMM to characterize vulnerabilities with knowledge con- tained in the parameters.

The detection phase is composed of the following steps:

1. Slice extraction: to extract slices from the source code, with each slice starting in an entry point and finishing in a sensitive sink. This is done by the slice extractor, which tracks the entry points and their dependencies until they reach a sensitive sink, independently if they are sanitized, validated and/or modified. The resulting slice is a sequence of tracked instructions.

2. Slice translation: to translate the slice into Intermediate Slice Language (ISL). We des- ignate the slice in ISL by slice-isl. During this translation, a variable map is created containing the variables present in the slice source code. ISL is a categorized lan- guage with grammar rules that aggregate in categories the functions of the server-side language by their functionality.

3. Vulnerability detection: to use the HMM to find the best sequence of states that ex- plains slice-isl. Each slice-isl instruction (sequence of observations) is classified by the model after the tainted variables from the previous instruction determine which emis- sion probabilities will be selected for the instruction to be classified. The classification of the last observation from the last instruction of the slice-isl will classify the whole slice as containing a vulnerability or not. If a vulnerability is detected, its description (including its location in the source code) is reported.