5.4 Software Updates Management
5.4.1 Packaging and Versioning
The dedicated packaging module (ad_management/packaging.py) is responsible for preparing installation and update packages from the SCION code repository. At the time of writing, the
5.4. Software Updates Management 25
SCION codebase repository has no specific versioning and releasing model as the project is still in heavy development. Therefore, it was decided to use the date and identifier of the last commit in the master branch to differentiate distinct distribution versions.
From the user’s point of view, he first opens the dedicated "Software updates" tab on the AD web page (Figure 5.6) and selects the version of the package he wants to operate with from the drop-down list. Then, the user can download the installation package (the button with the "Download the update" label) or install it to the AD node remotely (the "Install the update" button).
Chapter 6
Evaluation
This chapter provides details on performance and security aspects of the implemented testbed, including database and web application load testing, authentication and authorization ap- proaches, and checking the server-side component with web vulnerability scanners.
6.1
Performance Evaluation
According to the CIDR report1 from 05.07.2015, 51053 autonomous systems were registered on
the Internet. As mentioned in Chapter 2, autonomous domains (AD) and autonomous systems (AS) are two similar concepts from both structural and administrative points of view, and therefore approximately the same number of ADs (50000) was taken as a reasonable upper bound for various types of performance evaluation. For comparison, as of 02.08.2015, the PlanetLab testbed consisted of 1353 nodes.
In order to generate large network topologies, the BRITE [15] topology generator was used. As a first step, topologies for entire ISDs were created by the BRITE software package in accordance with the Barabási–Albert model [5] and saved in BRITE format. Then, the adapted SCION- specific module converted each BRITE file to the SCION topology format, additionally marking several ADs as core and adding links between cores of different ISDs.
All experiments were run on the following hardware:
• Operating system: 64-bit version of Ubuntu Linux 14.04 with kernel 3.13.0-55 • CPU: Intel Core i5-4570 3.2 GHz with 4 cores
• RAM: 16 GB
1http://www.cidr-report.org/as2.0/
6.1. Performance Evaluation 27
Routers Page load time SQL queries SQL execution time
25 2159 ms 60 27 ms
50 3972 ms 110 44 ms
100 7532 ms 210 79 ms
Table 6.1: AD details page performance: no select_related() optimization (50000 ADs) .
Routers Page load time SQL queries SQL execution time
25 462 ms 10 8 ms
50 603 ms 10 8 ms
100 888 ms 10 8 ms
Table 6.2: AD details page performance: with select_related() optimization (50000 ADs)
6.1.1
Database Performance
PostgreSQL2 (version 9.4.4) was employed as a primary database for the web application be- cause of its high performance, reliability and good integration with Django.
Django Debug Toolbar3 was used to gather necessary metrics such as total page rendering time,
number and execution time of performed SQL queries, etc.
Django has a built-in object-relational mapping (ORM) layer which saves developers from writing raw SQL code by providing a database-agnostic abstraction around all operations with the database. A programmer manipulates objects in object-oriented fashion using his language of choice (Python in the present case), while the ORM layer translates all high-level calls and methods to the specific query language of the underlying database. However, while the ORM abstraction facilitates development and improves security by exposing only a high-level interface to the database, the actual implementation of ORM might not always execute well-optimized code in terms of number of queries and their complexity. Therefore, a certain amount of manual tuning is required in order to achieve high performance.
The following subsections describe several optimization techniques which helped to eliminate redundant SQL queries and visibly improve the overall page load speed. The measurements were obtained for the scenario with 50000 loaded ADs, unless otherwise stated.
28 Chapter 6. Evaluation
Optimization 1: AD Details
The first web page which needs to load fast is the AD detail page, where all infrastructure elements are displayed along with their status. Assume the AD contains several border routers. For a given object ad which represents the AD, routers’ data can be fetched with one instruction, which looks as follows:
routers = ad.routerweb_set.all()
Behind the scenes, one SQL query is executed to fetch basic router information such as name, address and interface ID, and this query usually takes form of a single SELECT statement with the basic WHERE clause which filters all the routers by the AD id number. However, the fact that a router belongs to a certain AD is represented by the foreign-key (one-to-many) relationship, which references the AD database record from the router tuple. Now assume that we want to display an ISD the router belongs to somewhere on the page. ISD is an attribute of the AD model, since every AD "knows" the ISD it is located into. This implies that in the default implementation at least one additional SQL query will be executed for each router which searches the AD table for the attributes of the certain AD. The described issue results in linear growth of number of issued queries with respect to the number of routers and considerably damages the performance.
To avoid the abundant number of queries the select_related() method is used, which in- structs the ORM layer to follow foreign-key relationships, selecting additional related-object data in a single JOIN operation. The refined code takes the following form:
routers = ad.routerweb_set.all().select_related()
For performance comparison of the two presented approaches refer to Table 6.1 and Table 6.2. The results clearly indicate that the number of processed SQL queries in the non-optimized case grows linearly with the number of routers; SQL execution time and page load time behave similarly. In the optimized case (Table 6.2) the application issues the constant number of queries (10) which are executed in constant time (8 ms). The page load time grows because of the increasing number of displayed elements, but the increase is not as rapid as in the non-optimized case.
Optimization 2: Routers List
The administration site has a page which lists all router objects with the proper pagina- tion. Similar to the previous case, redundant queries were made if no optimization steps
2http://www.postgresql.org/
6.1. Performance Evaluation 29
Page load time SQL queries SQL execution time Non-optimized 7535 ms 204 93 ms
Optimized 431 ms 4 22 ms
Table 6.3: Routers list performance and the list_select_related optimization Page load time SQL queries SQL execution time Non-optimized 28312 ms 7 110 ms
Optimized 252 ms 7 5 ms
Table 6.4: Router details performance and the raw_id_fields optimization
had been taken (see Table 6.3). For this case, a special class variable exists in Django to avoid query pollution. This variable (list_select_related) must be set to True in the PrivilegedChangeAdmin class to inform Django to fetch the related objects in the same query. The applied optimization resulted in the constant number of queries as well as drastically reduced page load time.
Optimization 3: Router Details
The router details page is another automatically generated page on the Django administration site which allows to edit router’s data: name, address, connection ports, etc. A router has two foreign key attributes: the host AD (one the router belongs to) and the neighbor AD. In the default, non-optimized implementation Django will create a form with "select" widgets for AD attributes (see Figure 6.1a). This in turn implies that Django will populate the widget with all registered ADs, regardless of their total number. As a result, for 50000 ADs the resulting load time was more than 28 seconds (results are presented in Table 6.4), even with a small number of actual queries.
As a solution, the input type was changed for the two foreign key form fields by adding the raw_id_fields attribute to the corresponding ModelAdmin class. This changed the form field to the "raw" type, where a user enters the id of the object it refers to or selects it from the dedicated window.
6.1.2
Django Performance
In order to evaluate performance of the web application, several key web pages with their corresponding functions were selected to measure response time. These pages are:
• AD details page. This page serves as the main starting point for all information and actions linked to a given AD. It is also one of the "heaviest" pages since it displays different
30 Chapter 6. Evaluation
(a) No raw_id_fields (b) With raw_id_fields
Figure 6.1: Router change form
types of information on one page, separating them using a tab-like user interface. AD details include all infrastructure elements which belong to the AD, all sent connection requests, and available software updates.
• Router change page. This page is a part of administration site, provided by Django, It has many fields, including two foreign key fields: the AD it belongs to and the neighbor AD. Load testing of this page will also demonstrate performance of the admin site, which is an essential part of the application.
• New connection request form. Sending connection requests is an important feature of the management panel, allowing new users to join the network. It also represents a modifying operation because a new connection request object is created and saved to the database. This action is initiated by a POST request, all previous requests being read-only and performed with GET HTTP requests.
Boom4 load generator was chosen to evaluate the response time. This tool is capable of running
a specified number of HTTP requests against a web page with a given concurrency level (i.e., a number of workers which make requests). After it finishes, Boom provides information about the fastest and the slowest responses, resulting throughput, and a response time histogram among other things.
Performance was measured in four cases that correspond to the different numbers of ADs loaded in the web application: 50 (10 ISDs, 5 ADs per an ISD), 500 (5 ISDs, 100 ADs each), 5000 (20 ISDs, 250 ADs each), and 50000 (20 ISDs, 2500 ADs each).
6.1. Performance Evaluation 31
Figure 6.2: Web application performance
By default, Django allows to run the basic built-in test server to interact with an application. To get meaningful results the setup similar to production environment was used: Gunicorn5 WSGI
HTTP server with 9 worker processes ((2 ∗ number_of_cores + 1) is a default recommended number of workers). By default, a Django application is started with debug mode turned on, which facilitates development and debugging by providing comprehensive error messages and additional runtime information. However, debug mode consumes significantly more memory and CPU time, and it is highly discouraged to leave it turned on in the production setting because of security issues. Therefore, debug mode was turned off in project’s settings.py file (DEBUG and TEMPLATE_DEBUG variables set to False) before running load tests.
To produce the results, a set of 1000 requests were issued against each of three pages. This was done three times, after which the median value was taken for every test case in order to diminish the influence of other factors (CPU/RAM usage by other programs, caching, etc.). Network latency was not taken into account.
The results are presented in Figure 6.2. The figure illustrates that an increase in the number of ADs has no effect on the router change page performance (the red line) which reached more than 90 requests per second (RPS). Actions with connection requests (the purple line) became slightly slower, dropping from 59 RPS for 50 ADs to 47 RPS for 50000 ADs. The most noticeable drop in throughput can be observed for the AD overview page: 90 RPS for 50 ADs
32 Chapter 6. Evaluation
decreased to 71 RPS for the case of 5000 ADs, and then to 32 RPS for 50000 ADs. Nevertheless, the web application was usable even for the large number of ADs, and there exist further ways to improve performance: various caching frameworks (Memcached6, Redis7), database tuning,
and further optimizing the Django code, to name a few.
6.2
Security Evaluation
6.2.1
Web Security
Since the management panel is designed to be a critical part of the SCION testbed infrastruc- ture, it must be secured against a wide range of potential attacks. The Django framework has built-in mechanisms which help to prevent attacks through XSS, CSRF, clickjacking, and SQL injection vulnerabilities among others, but even the proper use of these techniques does not guarantee full protection against security risks.
General Security Policies
As a major general requirement, the web application must be accessible only via HTTPS site- wide with a valid certificate in order to avoid eavesdropping and man-in-the-middle attacks. At least version 1.2 of the TLS protocol should be used to prevent usage of weak cipher suites and attacks against previous versions of TLS and SSL, such as BEAST [1] and POODLE [16]. It is highly recommended to enforce several additional HTTP policies by properly configuring the web server:
• The HTTP Strict Transport Security [13] policy must be enforced in order to make SSL downgrade attacks and subsequent man-in-the-middle attacks more difficult. The mechanism is usually implemented with the special header the web server sends to the client. The contents of the header informs the client to use HTTPS exclusively for a specific period of time, thus protecting the user from eavesdropping and active network attacks.
• HTTP Public Key Pinning [12] is another security mechanism for HTTP which protects users and websites from impersonation attacks, when a malicious certificates might be issued by a compromised certificate authority. The user receives a special response header from the server which contains hashes of the website’s authentic certificate together with a period of validity. During this period the client should connect to the server using only the certificates whose hashes match the values from the received header.
6http://memcached.org/ 7http://redis.io/
6.2. Security Evaluation 33
In order to secure the server where the application is deployed, additional security measures must be taken. These measures include a properly setup firewall that block all incoming and outgoing connections, carefully configured network services such as SSH and FTP, and intrusion detection and prevention systems (IDS/IPS). The latter can monitor the system and the network for suspicious activity and promptly alert administrators in case of security incidents.
Security Scanners
In order to check the web panel for the presence of common vulnerabilities and misconfiguration errors, the application was scanned by two security scanners:
• Nikto8 (version 2.1.5)
Nikto is a web server scanner which performs exhaustive checks for vulnerabilities and server configuration.
The single noteworthy warning produced by Nikto indicated that anti-CSRF cookies, which are used to protect users from CSRF vulnerabilities, did not have the HttpOnly flag set. This was fixed by adding the corresponding directive to application settings in order to reduce the attack surface.
Apart from that, Nikto scan of the SCION web application produced no relevant warnings or errors.
• w3af9 (version 1.7.2)
The w3af (Web Application Attack and Audit Framework) project aims to create a pow- erful framework for auditing and securing web applications.
w3af offers several profiles (sets of vulnerabilities or potential configuration issues) to run the scans with. Two profiles were checked: OWASP_TOP10 (basic vulnerabilities from the OWASP Top Ten List) and full_audit (more comprehensive one).
Only one false-positive about the potential SQL injection on the login page was reported.