SQLi module design choices

SQLi design choice 1: have a database system to use

Since we are testing for SQL injections, we need to use a live database system during the testing. If the

framework supports multiple database systems, then you should test it with all of them. Sanitization code can depend on the database system and the interfacing module as well. However, testing with all databases might be too much work. If this is the case, it is also possible to test it only with the most common database system(s). Also the specific version of the database system could have an influence. In general, you should use the newest stable version that is supported by the framework. This is because we want to find security vulnerabilities in the framework, not in the database system and the newest version likely has the least security vulnerabilities itself.

SQLi design choice 2: include base sanitization methods and CRUD functionalities

The database interface supplies functionalities to interact with the database. Basically these functional-ities perform a particular kind of query on the database. We can categorise these functionalfunctional-ities on the kind of action they perform on the database. We make a distinction between standard CRUD database queries and non-CRUD queries. Examples of the first category are simple “INSERT INTO ...”, “SELECT ... FROM ...” queries. Examples of the latter category are database management queries (“CREATE DATABASE ...”) and user management queries (“CREATE USER ...”). In live web applications queries of the first category are by far the most common. In fact, queries of the second category are so rare, that we choice not to include them in the benchmark.

Except for the database interface functionalities for the standard CRUD actions, frameworks often have the possibility to execute custom SQL queries as well. Usually the framework supplies sanitization helper methods to secure these queries too. We call these methods the base sanitization methods and they should be included in the benchmark as well.

In some frameworks it is possible that functionalities can be combined, for example to build a query piece by piece². If this is the case, then these functionalities should not only be included separately, but also in (valid) combinations with each other. Whether this is the case and what the valid combinations are that should be tested, is something the benchmark initiator should determine, since it is framework specific. It could be that testing all possible combinations is infeasible. In that case it could be an option to only test the most probable combinations.

SQLi design choice 3: submodules for base sanitization methods and each CRUD action As discussed above we only include standard CRUD functionalities in the benchmark. However, each of the four CRUD actions are very particular and require a somewhat different approach to test. Therefore, we choice to distinguish between each CRUD action, and treat them as separate submodules in the benchmark. This has the additional advantage that we can easily enable/disable parts of the benchmark in order to focus the testing when running the complete module takes to much time. Another benefit is that some frameworks also make this (or a similar) distinction in their architecture to improve code cohesion. Therefore, there might be a connection between the modules of the benchmark and the modules of the framework, which makes it easier to track the responsible code once an SQL injection is found.

If the framework also includes base sanitization methods, as explained in the previous design choice, then these should be included as well in a separate submodule. Therefore, we have a total of five possible submodules, one for each CRUD action and one for the base sanitization.

SQLi design choice 4: enable full error reporting

The benchmark should enable full error reporting and all SQL errors that occur during the execution of queries must be displayed. This is actually already covered in global benchmark requirement 9, but we want to stress its important specifically here. This is because of the fact that dynamic scanners usually check for SQL injections, by checking responses for common database error strings. Often they also provide methods for detecting blind SQL injections (for example by differential analysis of the response, or by performing a timing attack), but these methods are somewhat less reliable and we want to make it the scanners as easy as possible.

SQLi design choice 5: have a suitable database table to operate on

The database interface functionalities should not only have a running database system to operate on, there should also be a database table in this system. The standard CRUD actions create, read, update or delete data in/from some table. This table should be created upfront before the tests can run and likely

2This is also the case in Ruby on Rails, where queries can be build by chaining so-called relation methods.

4.4. SQL injection module design

before the tests can be implemented, since the implementation depends on the columns in this database table. We should determine what this table should look like, i.e. what columns there are in this table.

We want our testing setup to be as complete as possible. Therefore, the table should include a column for each supported data type. To determine the supported data types, we should determine what kind of framework we are dealing with. There are three options:

1. The frameworks generates the database definition:

In this case the supported data types of the framework should be used. If the database system also supports other data types, then this will not matter, since (in normal use cases) these will not be used. If the database system does not support a data type supported by the framework and the framework does not use a fallback, then this data type should be skipped, since the application will not function otherwise.

2. The framework generates code based on the database definition:

In this case the supported data types of the database system should be used. If the framework also supports other data types, then these should be skipped, since it is not possible to test them with this database system anyway. If the framework does not support a data type supported by the database system, then it could be interesting to still explore this type and see how the framework handles it. In this case it should either use a safe fallback or not function at all.

3. The framework does not directly use the database definition:

This is usually the case for non-Active Record frameworks. There is no direct coupling between the database and the code. In this case you should test all the supported data types of the framework with all the relevant data types of the database system. For example, if the framework has a float data type, and the database system has both a float and a decimal data type, then both should be tested. The initiator of the benchmark should carefully decide which combinations of data types are relevant.

We do not only create the database table to operate on, but also prefill it with some records containing random data. We do this because some of the functionalities need data to operate on. For example in order to update a record in the database, there must be a record in the database that we can update.

For other functionalities (for example functionalities that perform a general DELETE query) it might not matter if there actually is any data in the table. However, even for these functionalities it is best to test them with data, since this is more realistic.

SQLi design choice 6: handle special arguments correctly

According to requirement 8 there should be guidance on the usage of the functionality’s arguments. For the SQL injection benchmark, most of the arguments will be represented as simple text fields. However, there are two types of arguments that are special and require a special representation in order to guide the dynamic scanners. These two arguments are:

1. ID arguments: some methods take an ID or list of IDs as argument, an example is a read method that takes an ID and returns the corresponding database table record. In order to provide the scanners some guidance on the expected data type it is best to represent this field as a select or multiple select field, prefilled with the IDs of the records currently in the database.

2. Condition arguments: some methods take conditions (usually represented as the WHERE part of the query) in order to limit the data on which they operate. Conditions are applicable for read, update and delete queries. The best way to test the conditions argument is to include a field for each possible condition. However, a condition can be complex, since it consists of a field, an operator and a test value. This gives many possibilities. Furthermore, these conditions can be chained together with other (logical) operators. Testing all this, might be too much work. Because of this, we are satisfied with testing one condition for each column in the test database table. We choose for the equality condition, since it is applicable on (nearly) every data type. If more conditions are given at once, then these should be chained together using the AND operator. This way the affected database rows will rather be limited, which turns out to be particularly useful later delete actions (since otherwise to many or even all rows could be deleted at once).