SEVENTH FRAMEWORK PROGRAMME
Theme SEC-2011.2.5-1 (Cyber attacks against critical infrastructures)
D5.1 Security Testing Methodology
Contract No. FP7-SEC-285477-CRISALISWorkpackage WP 5 - Vulnerability Discovery
Author Marco Caselli, Frank Kargl
Version 1.0
Date of delivery M12
Actual Date of Delivery M12
Dissemination level Public
Responsible UT
The research leading to these results has received funding from the European Community’s Seventh Framework Programme (FP7/2007-2013) under grant agreement n°285477.
The CRISALIS Consortium consists of:
Symantec Ltd. Project coordinator Ireland
Alliander Netherlands
Chalmers University Sweden
ENEL Ingegneria e Innovazione Italy
EURECOM France
Security Matters BV Netherlands
Siemens AG Germany
Universiteit Twente Netherlands
Contact information: Dr. Corrado Leita 2229 Route des Cretes 06560 Sophia Antipolis France
e-mail: [email protected] Phone: +33 673 41 91 27
Contents
1. Introduction 6
1.1. General Classification . . . 7
1.2. Standardization . . . 9
1.3. Testing Critical Infrastructures . . . 10
2. Standard 12 2.1. OSSTMM . . . 12 2.1.1. General information . . . 12 2.1.2. Overview . . . 14 2.2. NIST SP800-115 . . . 19 2.2.1. General information . . . 19 2.2.2. Overview . . . 22 2.3. ISSAF . . . 29 2.3.1. General Information . . . 29 2.3.2. Overview . . . 31 3. Requirements 37 3.1. Approach . . . 37 3.2. Questionnaire . . . 38 A. General information . . . 38
B. Security requirements, design, and regulations . . . 40
C. Testing and methodologies . . . 41
D. Research . . . 46 E. Follow up . . . 46 3.3. Further analyses . . . 46 4. Methodology 50 4.1. Overview . . . 50 4.2. Workflow . . . 51 4.2.1. Main Workflow . . . 51 4.2.2. Workflow phases . . . 53
Abstract
Existing security testing methodologies and tools cannot be applied directly to critical environments, due to a number of safety and availability requirements. One of the goal of WP3 is to understand what assessment processes we can exploit in Industrial Control System (ICS) environments and what are the constraints given by industrial components and infrastructures. This document provides an overview of assessment methodologies used in standard IT. We analyze these methodologies to investigate their applicability in an ICS environment. We are also conducting a survey among stakeholders in order to detail existing best practices adopted by companies to test their infrastructures and discover vulnerabilities in their systems. From this we are proposing a new methodology tailored to support security testing in Critical Infrastructure environments. As the evaluation of the feedback and the methodology is requiring more work than originally foreseen and as our on-going investigations are also bringing up new aspects not foreseen in the original proposal (e.g., incident handling), this deliverable should be regarded only as a first version of D5.1 that we intend to extend in a version 2 either after year 2 or at the end of the project in order to address these additional aspects.
Security testing involves various techniques which complement security engineering, i.e., design and implementation of secure systems. In contrast, security testing implies testing of those systems with regards to their security properties. Security testing includes analysis-centered activities like a code review but also practical tests to penetrate a security system and find vulnerabilities and implement attacks. The latter is often termed penetration testing.
Per definition, a penetration test is a simulation of an attack aimed at assessing the security of a computer system or a network. Simulation refers to the fact that the test is conducted by the system owner or on his request. This test has multiple phases and allows to highlight infrastructures’ and protocols’ weaknesses. During a penetration test, the penetration tester tries to discover and sometimes exploit known or unknown vulnerabilities in order to obtain useful information or to access targeted hosts. At the end, all security problems discovered are reported to the system owner together with an assessment of their impact on the system. The analysis should also propose technical or organizational mitigations to identified problems.
In its most simplistic form, a penetration test only involves running some automated security scanner, like, e.g., Nessus [1]. However, it is generally agreed that serious pene-tration testing typically involves also manual test planing, preparation, and conduction to be able to flexibly react to the system under test. In such a case, the penetration tester should follow some structured methodology to ensure valid, reproducible, and well documented results.
The development and use of such manual penetration testing methodologies is moti-vated by several reasons. First of all, there are numerous vulnerabilities that may be difficult or impossible to detect with automated tools. Network or application vulnera-bility scanning software are nowadays highly sophisticated. However, sometimes, high profile systems and infrastructures need a comprehensive and systematic approach to be compromised which cannot be achieved by automated tests. This is, e.g., related to the detection of high-risk vulnerabilities resulting from a combination of low-risk ones. As multiple minor weaknesses are exploited in a particular sequences, a penetration test may reveal all the possibilities identifying the possibly dangerous combinations.
More importantly than identifying vulnerabilities, determining the feasibility of a par-ticular set of attack vectors is another important target of a penetration test. As having
1.1. General Classification
theoretically unassailable systems seems impossible in practice, companies and organiza-tions are interested in running infrastructures that are “practically” secure. Therefore, the analysis becomes a way to rank vulnerabilities both by severity and realizability.
The potential business impact is a further valuable indicator of importance for a vul-nerability. Conducting practical attacks also aims at properly identifying necessary at-tacker effort which is a prerequisite for correct risk estimation. Assessing the magnitude of economical and operational problems related to a successful cyber attacks is usually the last phase of analysis of a penetration test. Based on the knowledge of business processes such analysis gives an accurate picture of the risk related to the infrastructure and to the work such infrastructure performs.
Finally, penetration tests are the experiments with which companies try out the ability of network defenders to successfully detect and respond to cyber attacks. Their capacity of reaction is measured during the test and evaluated at the end of it. It will also provide evidence to support increased investments in security personnel and technology or it will suggest modifications in methods and strategies are the output of such kind of assessments.
1.1. General Classification
Penetration tests can be performed in a broad variety of ways. The following criteria provide some orientation and classification.
The amount of information known by the fake attackers is one of the main discrim-inating factor of penetration tests. The simplest classification describes three different types of tests: white-box,black-box, and gray-box.
White-box penetration testing outlines a situation in which fake attackers have full knowledge of the system under attack. The goal of a white-box penetration test is to simulate the presence of an expert malicious insider. In some special cases such insider has also basic credentials for the target system. Among the information known to the fake attacker we can have: network diagrams, domain names, IP addresses, but also phone numbers, e-mail addresses. White-box testing of applications usually allows intruders to have also programs’ source code. White-box test design techniques include [2]:
Control flow testing
Data flow testing
Path testing
Statement coverage
Decision coverage
Black-box penetration testing is the opposite of the previous one. This method examines infrastructures’ and applications’ functionalities without knowing anything about them. In general, the tester is “aware of what the software is supposed to do but is not aware of how it does it” [3]. Thus, specific inputs (e.g network packets, shell commands, etc.) return specific outputs without the intruder knowing how the mechanisms within the system work. The goal of a black-box penetration test is to simulate a typical external hacking or also cyber warfare scenario. Typical black-box tests design techniques include [4]:
Decision table testing
All-pairs testing
State transition tables
Equivalence partitioning
Boundary value analysis
Finally, Gray-box penetration testing is a combination of white-box and black-box meth-ods. A gray-box tester partially knows the internal structure of the system under attack. This knowledge can include network topology, installed applications or also more in deep information like used algorithms and data structures. Since any combination of knowl-edge owned by the testers falls in the domain of gray-box penetration testing this method is the most widely used, especially in the field of web applications. Gray-box testing techniques include [5]:
Matrix Testing
Regression testing
Pattern Testing
1.2. Standardization
Other ways to characterize penetration tests include the type of system access (ex-ternal network, in(ex-ternal network, physical access), whether a test is conducted in pure physical ways or whether social engineering attacks are also performed, or whether test-ing includes performtest-ing (possibly disruptive) attacks or whether such attacks should only be identified but not performed. The specifics of a test need to be carefully defined in the test plan.
1.2. Standardization
In this last decade the interest in penetration testing techniques has increased. Com-panies, organizations and governments provide numerous services on the Internet and need to assess the security of their systems. Given the proliferation of different pene-tration testing methods, some organizations and standardization bodies decided to pro-pose comprehensive and usable methodologies. While many penetration testers follow a “home-brewed” self-defined testing approach, following standardized approaches has some general benefits.
A standardized and generally accepted penetration methodology provides a certain confidence that no important steps or aspects of a test are forgotten by accident. As tests should be repeated after significant changes to the system under test, following the same methodology also provides the possibility to compare multiple tests to identify whether a system got more secure after mitigating measures have been taken. A standardized approach may also be required in cases where security tests are required from a legal or insurance perspective. A standardized approach and common reporting forms also allows to compare results with results from other installations. Finally, accepted standards simplify negotiations between a security tester and clients about the scope of such tests. So there are a number of good reasons to look into such standardized and commonly agreed approaches.
Over the years, many standardized methodologies have been proposed. From most general to system-specific ones, both governments and private organizations have pro-posed different ways to approach penetration tests. Among the most well-known pene-tration testing techniques are: OSSTMM (Open Source Security Testing Methodology Manual), NIST SP800-115, ISSAF (Information Systems Security Assessment Frame-work), OWASP (Open Web Application Security Project), and PTES (Penetration Test-ing Execution Standard). Later in this document, we are goTest-ing to give a broad overview of some of these approaches.
In addition to methodologies, different kind of systems exist to assess specific contexts of security testing without a structured approach. Companies can decide not to follow a
methodology but just perform a light assessment using different methods. In this cases they can exploit several guidelines and toolkits developed for numerous and different purposes. According to [6] toolkits “implement and bundle in a convenient package a set of testing techniques, usually aimed at discovering specific classes of security problems”. Guidelines instead “organize the process of security testing, by collecting sets of best practices, comprehensively listing items to be tested, and structuring any other kind of useful advice”. Finally, there are also collections of penetration testing tools such as Backtrack Linux or metasploit that are regularly used by penetration testers.
1.3. Testing Critical Infrastructures
Despite the recent interest for penetration testing methodologies, none of the proposed methodologies specifically considers industrial control systems and critical infrastruc-tures despite the fact that penetration testing is also performed in such environments on a routinely basis.
We argue that standard methodologies should not be applied there without special consideration. These systems and the networks that interconnect them show many dif-ferences compared with standard IT networks. Since critical infrastructures control and automate real world processes or equipments which are potentially dangerous for hu-mans, any activity different from the main purpose including penetration testing should be limited. So just running a full Nessus or Metasploit test in such an environment could have disastrous consequences.
There are three main concerns that make penetration testing on critical infrastructures challenging. As introduced in the previous paragraph, any penetration attempt can have a consequence in the real world. What follows is an interesting example of such concerns. In a real incident, a gas utility conducting a penetration test on its corporate network performed this in a careless manner affecting some of the DCS systems. During the test this DCS system was locked up by mistake and the utility was not able to send gas through its pipelines for four hours leading to a large economic loss [7]. The strong relation between the system under test and physical components that people depend on is the first important difference with respect to standard IT. This implies that any CI penetration testing methodology needs to include safeguards to protect from such failures.
Another important difference concerns the devices used in Critical Infrastructure, these systems usually have very limited resources compared to devices found in normal IT environments. The longevity of network components is greater and the cycle of replacement of devices may also be in the order of decades. This implies that devices
1.3. Testing Critical Infrastructures
may be out of service, no patches may be available, and generally proposing mitigation strategies may take different approaches. This also means that critical infrastructures can be more easily overloaded with respect to other IT systems. A penetration testing methodology has to take care not to override the normal operation of every component in the ICS unless identifying DoS attacks is specifically in the scope of the test.
Finally, due to the size of most of the critical infrastructures, a penetration testing has always to check if localized attacks have effects in different parts of the network which may be hard to assess due to indirect effects. At the end of a test the identification of a vulnerability is followed by the calculation of the damage that it can cause. Linking these two elements in a system with thousands of components requires the assistance of experienced personnel both in the IT field and in the field related to the industrial process under control. As a consequence, penetration tests cannot be conducted by security experts in isolation.
We think that a penetration testing methodology for critical infrastructure must ex-plicitly describe how to deal with the previous issues. Moreover, it has to identify specific methods, tests, and tools already used in standard penetration testing techniques and explain how to tune these in order to be suitable for industrial control systems. Finally, such an ICS penetration testing methodology must provide – where necessary – new instruments and customized methods to test IT and networking components not found in standard ICT (e.g., Process Control Units, Remote Terminal Units, field networks, etc.).
In this chapter, we are going to review multiple penetration testing methodologies pro-posed by standardization bodies and other organizations. The aim is to provide a broad overview over the state of the art before defining specific requirements for CI penetra-tion testing and then proposing an own CRISALIS penetrapenetra-tion testing methodology for Critical infrastructures that is distilled from those existing approaches.
2.1. OSSTMM
2.1.1. General information
The Open Source Security Testing Methodology Manual (OSSTMM) [8] is an open stan-dard methodology for security tests. Developed by Pete Herzog at the end of 2000 as an ethical hacking framework, it has rapidly grown to become a comprehensive method-ology to assure security at operational level. Version 3, released in 2008, encompasses tests for every security aspect: from personnel qualification to physical security, from control of communication to electronic systems safety. As every standard methodology, it is designed to be consistent and repeatable. Moreover, it is openly available and thus allows a free dissemination and free use.
Figure 2.1 shows a comparison among OSSTMM and common security tests in terms of accuracy and thoroughness. By its nature, the methodology can be adapted to any kind of audit operation (e.g. penetration tests, ethical hacking, security and vulnera-bility assessments, red/blue teaming, etc.). The primary purpose always remains the description of a security examination path and the definition of a way to consistently correlate test results.
The usage of the OSSTMM allows the auditor to perform a validated set of tests and to qualify its examination as a “certified OSSTMM audit”.
Differently from other methodologies, mitigations to security flaws are not required as part of an OSSTMM audit. Recommendations may well be part of the final results, but common operational situations often imply that auditors have a limited view and knowledge of the client environment. In these cases it is difficult to propose proper and feasible solutions. For this reason solutions are usually avoided in final reports but should we worked out together with the system owner after the test. This is an aspect
2.1. OSSTMM
Figure 2.1.: OSSTMM comparison with common security tests (from [8])
that is well in line with our findings from the previous chapter as it will also apply to CI.
An important aspect of the OSSTMM is the compliance with general security policies. The methodology is developed taking into account major legislation and regulations. Furthermore, OSSTMM defines three different types of compliance and a set of rules to deal with all of them. The three types of compliance defined in the methodology are:
Legislation: enforces the security test to be compliant with region/country legis-lation. The lack of this requirement could lead to criminal charges.
Regulation: enforces the security test to be performed accordingly to standard regulation within industry sector. A failure in this task could lead to a dismissal of the company from the group.
Policy: enforces the security test to be done according to business and organization policies. Violation could cause a dismissal of the responsible from the company. OSSTMM provides a list of national and international regulations already proved to be compliant to the proposed methodology.
2.1.2. Overview
Before describing the workflows, the OSSTMM introduces several concepts to outline what kind of security tests can be performed and what the scope of such tests can be.
Figure 2.2 presents the six different security test types defined by OSSTMM. Possible security tests are however not limited to these cases and can be tuned to more specific requirements.
Figure 2.2.: OSSTMM security test types map (from [8])
Blind: this type of security test describes a situation in which the attacker does not know anything about the target. On the other side, the defender knows everything about the audit operations and it is prepared to take countermeasures. A blind security test is primarily used to evaluate attacker skills.
Double Blind: as before the attacker does not know anything about its target but, this time, the target is not notified in advance about the security test. A double blind audit is used to test both the skills of the attacker and the preparedness of the defender.
Gray Box: in a gray box security test the attacker has limited knowledge of the target and can better engage the defender. The defender still has the advantage of full knowledge about the audit in progress. As in the blind security test, this case is mostly used to evaluate attacker’s skills.
2.1. OSSTMM
Double Gray Box: in this type of security test both the attacker and the defender have limited knowledge about their counterparts. Such situation is used to evaluate both the contenders. With respect to double blind this test increases the level of accuracy and pervasiveness.
Tandem: in the Tandem security test the attacker has full knowledge of the target and the defender is fully prepared to bear the test. Such situation is used to improve the thoroughness of the audit as the attacker has a full view of all tests and their responses.
Reversal: as in the previous situation, the attackers knows everything about the target while the defender knows nothing about its contender. The purpose of this test is to evaluate every operational aspect of the defender.
Once the test type of interest is identified, it is necessary to identify the scope of the audit. OSSTMM divides such scope into three different channels: COMSEC (com-munication security), PHYSSEC (physical security), SPECSEC (spectrum security). A thorough security audit would require testing all three channels but usually the tests are tuned on more specific needs. For this reason, the methodology makes a further parti-tioning, addressing these three channels into five logical sections. Channel PHYSSEC is divided into “Human” and “Physical”, channel SPECSEC contains “Wireless Commu-nication”, and channel SPECSEC is divided into “Data Networks” and “Telecommuni-cations”.
What follows is a description of the five logical sections:
Human: concentrates on the human element and its communication processes.
Physical: focuses on physical security testing and comprises tangible elements of security where interaction requires physical effort or an energy transmitter to manipulate.
Wireless Communications: concentrates on security tests on wireless electronic communications and signals.
Data Networks: comprises tests on electronic systems and data networks.
Telecommunications: focuses on digital/analog telecommunications where in-teraction takes place over established telephone network lines.
The full methodology is divided into seventeen modules. Every module defines a specific target and several tasks that are needed to achieve it. Each module has an input
and an output. The input represents the information needed to perform each task while the output is the result of the accomplished tasks. Figure 2.3 shows the methodology flow.
OSSTMM uses the same workflow for all the channels. The seventeen modules and the methodology flow apply in the same way to the five logical sections. However, while the methodology remains the same, each channel will differ in the proposed tasks.
The methodology is divided into four phases: Regulatory Phase (modules 1 to 3), Definitions Phase (modules 4 to 7),Information Phase (modules 8 to 13) and Interactive Controls Test Phase (modules 14 to 17). Each phase faces a different level of depth in the audit. The regulatory phase focuses on understanding the audit requirements, scope and constraints. The definitions phase improves the description of the scope and of the inner interactions and processes. The information phase uncovers misplaced and mismanaged information flows in the target. Finally, the interactive controls test phase focuses on practical penetration testing. This is often the final phase of a security test. This ensures that practical attacks do not affect less invasive tests.
In what follows we briefly describe the focus of every module.
Posture Review: it is used to define the scope and identify what tests must be performed. This module focuses on rules, norms, regulations, legislations and policies applicable to the target.
Logistics: it defines which limitations the audit process has. It relates to interac-tion constraints such as physical distance, communicainterac-tion speed, etc.
Active Detection Verification: it takes into account any practical restriction imposed against performing interactive tests on the targets.
Visibility Audit: it determines available targets within the scope. This module provides the first in depth description of the targets and how they interact with the scope.
Access Verification: it measures the breadth and depth of interactive access points. It verifies if such access point to the target exists and if it requires authen-tication.
Trust Verification: it identifies trust relationships from and between the targets. This module verifies if targets accept interaction freely and without credentials.
Controls Verification: it measures the effectiveness of process-based attributes like non-repudiation, confidentiality, privacy and integrity.
2.1. OSSTMM
Process Verification: it verifies the existence, effectiveness and maintenance of security levels defined for the targets.
Configuration/Training Verification: it explores the default conditions under which targets operate regularly to understand their intents, business justification and reasoning.
Property Validation: it checks the use of illegal or unlicensed intellectual prop-erty or application within the targets.
Segregation Review: it checks the amount of personally identifiable information available depending on the applied rules and legislations.
Exposure Verification: it searches for freely available information which uncov-ers indirected or unwanted visibility of the targets.
Competitive Intelligence Scouting: it searches for freely available information which could harm or adversely affect the target.
Quarantine Verification: it determines the effectiveness of authentication in terms of black and white listing quarantines.
Privileges Audit: it measures the impact of misusing credentials and privileges and the consequences of unauthorized escalations of privileges.
Survivability Validation/ Service Continuity: it checks the resistance of the target to excessive or adverse changes.
Alert and Log Review/End Survey: it is a review of the performed audit activities.
The outcome is a comprehensive assessment of the tested channels. Results are pre-sented throughRisk Assessment Values (RAVs) that are a set of security metrics pro-viding an overview on the state of the channels (e.g. measurement of the attack surface, the amount of uncontrolled interactions with a target, etc.). To finally measure the thoroughness of the test performed, the OSSTMM suggests to conclude the work with a Security Test Audit Report (STAR). This step requires to record in the appropriate form the following information:
Date and time of test
2.2. NIST SP800-115
Auditors and analyst involved
Test type
Scope of test
Test Index
Channel tested
Test Vectors
Verified test and metrics calculations of the operational protection levels, loss con-trols and security limitations
Knowledge of which tests have been completed, not completed, or only partially completed, and to what extent
Any issue regarding the test and the validity of the results
Test error margins
Any processes which influence the security limitations
Any unknowns anomalies
Overall, OSSTMM provides a very comprehensive framework that addresses a large variety of practical aspects. It is specifically useful to ensure completeness of our later approach. However, it is also relatively generic and is not going into details of specific systems and is not addressing the CRISALIS scenarios at all.
2.2. NIST SP800-115
2.2.1. General information
The National Institute of Standard and Technology (NIST) has designed its own security assessment methodology in the Special Publication 800-115 [9]. The purpose of this publication was to provide guidelines for organizations on planning, conducting and evaluating information security testing. Even if the overall goal is to propose an overview of main key elements of technical security assessments, NIST SP800-115 provides also practical recommendations and technical information relating to penetration tests.
NIST defines the security assessment as “the process of determining how effectively an entity being assessed meets specific security objectives”. From an high-level point of view, there are three possible ways to accomplish such target:
through Testing, by exercising the assessment objects comparing actual and ex-pected behaviors
throughExamination, by analyzing the assessments objects and understand their functioning
through Interviewing, by conducting extensive discussions with organizations and experts to identify possible problems
A methodology that implements one of the previous points has to contain at least three different phases:
Planning: this phase is necessary to collect all the information needed for the as-sessment. This concerns elements like assets, threats, policies in use, etc. Moreover, the assessment should have a final goal and a set of objectives to work towards.
Execution: this phase concerns the operational part of the assessment. The Exe-cution implements operations of environment discovery and analysis, vulnerability identification and result validation. Practical activities for this phase obviously differ from case to case. However, the main goal is to bring to light problems at technical and organizational level.
Post-Execution: this phase describes all the analysis aim at identifying vulner-ability causes and mitigation strategies. The development of a final report is the main output of the Post-Execution.
As already discussed, NIST SP800-115 offers an overview of security assessment key points. However it suggests several practical techniques to implement penetration test-ing. According to the authors, the most commonly used pentesting techniques can be grouped in the following categories:
Review Techniquesdefine how to examine and analyze assets and their security policies.
Target Identification and Analysis Techniques define how to identify assess-ments targets and how to describe their properties.
2.2. NIST SP800-115
Target Vulnerability Validation Techniques: define how to verify the pres-ence of vulnerabilities within the targets and how to test their security levels. The NIST methodology focuses on how these techniques can be used together, but does not provide a list of technical activities to be used in any specific circumstance. In addition to practical penetration testing techniques, the methodology proposes several non-technical methods that can be used to enforce the assessment (e.g. physical security testing).
Finally, NIST SP800-115 concentrates on how singular tests and final reports can be compared with each others. The possibility to compare two security assessments relies on the correct representation of the Testing Viewpoint. Every test can be performed from different viewpoints. This implies different information owned by the auditors or different physical location of the pentesters. Results of comparisons among different assessments always need to consider the kind of testing. NIST SP800-115 defines four testing types:
External Security Testing: this type of testing is conducted from outside the organization’s security perimeter. The goal of this security test is to reproduce the view of an external attacker and to draw the attention to vulnerabilities already visible from outside the company premises (e.g. from the Internet). External testing always begins with discovery techniques aiming at examining organization public information.
Internal Security Testing: differently from the previous one, this kind of testing is performed within an organization’s perimeter (e.g. internal network). The purpose of the Internal Security Testing is to impersonate a trusted insider or also an attacker that has penetrated external defenses. This test allows pentesters to have some level of access to the network and, thus, to internal information. These elements make it possible to assess internal security mechanisms.
Overt Security Testing: also known as White Hat testing, defines an Internal or External Testing in which pentesters have a full knowledge of organization’s systems and processes. Company staff is also aware of the test and it usually tries to limit the testing impact. This kind of test is often used as company training.
Covert Security Testing: also known as Black Hat testing, uses an adversarial approach by not giving any knowledge about the company to the pentesters. In this case, company IT staff is usually also not aware of the test and its response is used for testing the technical and organizational security controls within the company.
2.2.2. Overview
In this section we describe in detail the three groups of techniques discussed before and the three phases into which the methodology is logically structured.
Review Techniques describe the process of passively gathering information related to the organization and the consequent analysis operations. This information regards both policies and equipments (e.g. systems, networks, etc.). Since these techniques are passive, they do not interfere with company operations and systems. For this reason, this is usually the first phase in every security assessment. Possible review techniques are:
Documentation Reviewexamines the technical aspects of used polices and pro-cedures. Security flaws or weaknesses in such elements can lead to missing or improperly implemented security controls. The results of the analysis can also be used to better tune the following techniques.
Log Review determines if important information and operations are properly logged and recorded. This is a necessary element to prove that all the systems are working according to the applied policies and procedures. Log analysis can show also misconfigured systems and malicious operations (e.g. attempted or successful attacks).
Ruleset Reviewchecks the collection of rules and signature-based security mech-anisms used to detect malicious operations or implement organization’s processes. Routers, firewalls and intrusion detection systems are the main targets of this anal-ysis. Router access control lists are checked to identify whether rules are no longer needed or enforce the refusal of unknown traffic. The review refines in depth fire-wall rules by checking unnecessary open ports and allowed services. Moreover, it searches for possible backdoors and malware. Finally, IDS rulesets are verified to be up to date and tuned to organization’s needs.
System Configuration Review searches for weaknesses in systems and applica-tions configuraapplica-tions (e.g. programs not enforcing security or non-compliant with security policies). This review usually concerns: unnecessary services running on servers, improper user account and password settings, etc.
Network Sniffing implements the passive monitoring of a data network. It can rely on communication flows, protocols’ decoding, and header/payload analysis. It is used to map organization communication infrastructures and gathering infor-mation about used systems, applications and services. It can also identify
unau-2.2. NIST SP800-115
thorized and inappropriate activities or capture unencrypted personal information (e.g. usernames and passwords).
File Integrity Checkingprovides a way to check file modifications. It is usually implemented through checksum.
Target Identification and Analysis Techniquesfocus on identifying active devices through open ports and running services and analyze them looking for vulnerabilities. NIST SP800-115 discusses several different techniques belonging to this group:
Network Discoveryconcentrates on discovering active and responding host on a network. Active techniques send various types of packets to the network waiting for responses. This system is usually an improvement of the passive analysis discussed in the Review Techniques as it is able to find hosts not currently involved in any communication. During this analysis, pentesters have to face firewalls and intrusion detection systems that usually identify many instances of scans. Active discovery generally produce network noise and communication delays.
Network Port and Service Identificationis usually performed in parallel with the network discovery. This check involves the use of port scanners able to un-derstand services and application running on remote hosts. Information gathered during the process allows pentesters to exploit OS fingerprinting tools. Such appli-cations guess operating systems working on the network and provide a description about their versions and updates.
Vulnerability Scanning, like the previous two operations, collects information about hosts in the network. Moreover, it also attempts to identify specific vulner-abilities on discovered hosts. Vulnerability scanners usually provide: a check on the compliance of security policies, a list of targets for the following penetration testing, and preliminary information on how to mitigate discovered vulnerabilities.
Wireless Scanning: as the number of wireless devices is increasing, these tech-niques are becoming more and more important. Wireless scanning techtech-niques can be further divided in: passive scanning, active scanning, bluetooth scanning, and device location tracking. An example of passive scanning involves the analysis of MAC addresses (e.g. vendor identification or white/black listing of devices). An active scanning can be used instead to verify authentication mechanisms and data encryption. Bluetooth security issues have been extensively discussed in litera-ture [10], [11]. Bluetooth scanning is more difficult than wireless one due to the its short range. However, a confirmation of compliance with organization security
requirements is needed to have a comprehensive security assessment. Finally, wire-less networks can be used to locate organization’s devices. This check enforces the reconstruction of the company’s network topology.
Target Vulnerability Validation Techniques uses the information found in the two pre-vious phases and further explore the existence of potential vulnerabilities. The target is usually not limited to identifying the vulnerability but also to show the security exposure by trying to exploit it. This procedure is often risky for the organization infrastructure and processes. For this reason is usually the last active work performed on the network. In this way, it will not compromises other tests. NIST SP800-15 defines and implements its penetration testing method in this phase. The targets of the penetration testing can be numerous. Pentesters are interested in understanding how well the system tolerates real attacks, identifying necessary countermeasures, and evaluating defenders’ abilities. Moreover, this attack simulation can show what level of skills attackers need to compro-mise organization’s systems. The proposed penetration testing is divided in four steps as show in Figure 2.4.
Figure 2.4.: NIST SP 800-115 penetration testing steps
ThePlanning step prepares the necessary information and background for the attack. The Discovery step integrates information already owned by pentesters with a new session of network scanning. With respect to the other phases, the attackers try to acquire information about: host names and IP addresses, employee names and other contacts, and system information (e.g. names and shared resources). In some cases also techniques as “dumpster diving” can be used to enforce the process. The Discovery step improves also the vulnerability analysis and finalizes the list of targets. TheAttack step is the core of every penetration testing. During the attack a continuous feedback
2.2. NIST SP800-115
is provided back to the discovery step as succeeding to access a system gives pentesters additional knowledge of the infrastructure under attack. Figure 2.5 shows an example of attack.
Figure 2.5.: NIST SP 800-115 attack example (from [9])
NIST SP800-115 organizes the most common vulnerabilities exploited during the At-tack step in the following categories:
Misconfiguration: misconfigured security settings (e.g. insecure default set-tings).
Kernel Flaws: security flaws in the core code implementing an operating system that can endanger the whole system.
Buffer Overflows: inadequate checks on input sizes. This misbehavior can cause arbitrary code been executed with the privileges of the running program.
Insufficient Input Validation: missing checks on the content of input strings, files, etc. Network applications without filters can run malicious commands (e.g. SQL injection).
Symbolic Links: as the operating system allows to change permissions related to files, a malicious user can trick the system by creating symbolic links and modifying permission passing through them.
File Descriptor Attacks: malformed file descriptors can grant access to the related files.
Race Conditions: an attacker can take advantage of high privileges processes by modifying their execution order and manipulate results.
Incorrect File and Directory Permission: incorrect permissions can cause various lacks of information (e.g. reading and writing password files).
The last step, called the Reporting, occurs simultaneously with the other three steps and records all the results. At the end of the test, pentesters report the identified vulnerabilities, provide a risk rating, and propose possible mitigation techniques for discovered weaknesses.
Security Assessment Planning provides information on creating an assessment policy, scheduling tests, identify the most suitable approach, and addressing possible problems. It also enforces assessments to be compliant to current regulations and legislations. According to NIST SP800-115 this planning activity should identify:
Organizational requirements assessments have to take into account
Roles and responsibilities
Adherence to already established methodologies
Assessment frequency
Documentation requirements
In this phase organizations have to choose assessment techniques and customize them to fit the requirements. Organizations should also carefully consider risks related to use such techniques on their infrastructures. The selection of auditors and the identification of needed skills is another fundamental step of the planning. Assessors experience can determine the success or the failure of the evaluation. Organizations can choose the type of assessment (e.g. External/Internal and Overt/Covert) and finally propose technical tools and resources available to the assessors.
2.2. NIST SP800-115
What is the scope of the assessment?
Who is authorized to conduct the assessment?
What are the assessment’s logistics?
How should sensitive data be handled?
What should occur in the event of an incident?
Security Assessment Execution phase highlights key points for assessors to consider throughout the execution phase.
Assessors have to be coordinated with various entities and stakeholders within the organization. Appropriate organization’s personnel should help and supervise the as-sessment for the whole duration of the activity. During operations assessors can face different kind of problems (e.g. technical, operational, political, etc.). These can in-clude:
Resistance: to the assessments, from entities within the organization (e.g. net-work administrators or end users).
Lack of Realism: that indicates the preemptive modification to system settings in order to make the infrastructure more secure to pass the tests.
Immediate Mitigation: that describes the adjustment and the resolution on-the-fly of weakness found during the assessment. This can disturb following tests and compromise the final report.
Time: time constraints are usually a problem as security assessments are often an exception more than a normal operation integrated in the development and deployment cycle.
Resources: organization can make a stand on providing and maintaining adequate resources for security assessments.
Evolving Technology: tools used for security assessments as well as used tech-niques can be out-of-date.
Operational Impact: even if a security assessment should prevent any impact on infrastructure functioning, there is always a chance of accidental problems. For this reason, an incident response plans should be already in place during testing.
Assessors can make some preliminary analysis on the assessment during the assessment itself. This preliminary analysis of the causes of the noticed problems becomes the input for the last phase of the security assessment. Most common problem causes can be organized in the following sets:
Insufficient patch management
Insufficient threat management
Lack of security baselines
Poor integration of security into the system development life cycle
Security architecture weaknesses
Inadequate incident response procedures
Inadequate training for end users and network/systems administrators
Lack of security policies or policy enforcement
Finally, thePost-Testing Activities concentrates on finalizing the analysis and propos-ing mitigation actions and recommendations. Security testpropos-ing reports should be used by the organization as follows:
As a reference point for corrective action
In defining mitigation activities to address identified vulnerabilities
As a benchmark for tracking an organization’s progress in meeting security re-quirements
To assess the implementation status of system security requirements
To conduct cost/benefit analysis for improvements to system security
To enhance other life cycle activities (e.g. risk assessment)
To meet reporting requirements
Overall, NIST SP800-115 also provides a well-structured and complete approach that goes into more technical detail than OSSTMM. From this perspective, it is interesting to discuss whether such details are necessary and helpful and make a methodology more applicable or whether they just inevitably restrict its use to certain use cases.
2.3. ISSAF
2.3. ISSAF
2.3.1. General Information
The Information Systems Security Assessment Framework (ISSAF) [12] is a peer re-viewed structured framework designed by the Open Information Systems Security Group (OISSG). The methodology defined by ISSAF covers all the aspects related to security assessments: from an high-level perspective (e.g. business impact and organizational models) to practical techniques (e.g. security testing of passwords, systems, network, etc.).
The framework is divided in four main phases structured in several working packages (named “activities”). The four phases are respectively: Planning, Assessment, Treat-ment, and Accreditation. Figure 2.6 shows ISSAF workflow.
Figure 2.6.: ISSAF framework overview (from [12])
whole assessment process. Before proceeding with the operational part, organizations have to motivate the assessment to substantiate the underlying concerns. Moreover, organizations have to prepare a business plan to cover costs and identifying available personnel and resources. The main steps in this phase are:
Information Gathering
Project Chartering
Resource Identification
Budgeting
Cash Flow
Work Breakdown Structure
Project kick-off
TheAssessment phase defines the approach needed to evaluate the Information Secu-rity Risks within an enterprise. This phase focuses on the risk assessment process and addresses every component involved. It is divided into two categories: Inherent Risk Identification and Controls Assessments. The first consists of the following activities:
Identification of Assessment entities: during this activity the organization spots the targets of the assessment (e.g. processes, assets, facilities, etc.).
Identification of Threats and Vulnerabilities: this activity describes how to search, list and describe organization’s vulnerability.
Impact Assessment: this activity measures the impact of an exploited vulnera-bility to the business of the organization
Likelihood Assessment: during this activity the organization evaluates the like-lihood that a vulnerability is exploited.
Control Assessment explores instead the following fields:
Evaluation of Legal and Regulatory Compliance: this activity evaluates the position of the organization with respect to current regulations and legislations.
2.3. ISSAF
Evaluation of Enterprise Information Security Policy: this activity focuses on understanding current organization’s approach on implementing and maintain-ing its security policies.
Evaluation of Enterprise Information Security Organization and Man-agement: this activity directly follows the previous one, as a review on how the Information Security is organized inside the company.
Assessment of Enterprise Information Systems Security and Controls: this activity is in charge of reviewing different security aspects of the organization. For example: physical security, environmental security, technical controls (e.g. network security, host security, etc.), social engineering, etc.
Evaluation of Enterprise Security Operations Management: during this activity the organization gains an understanding of the risk of specific operations processes. The operational areas involved in this operations are: Capacity Man-agement, Vulnerability ManMan-agement, User ManMan-agement, etc.
Evaluation of Enterprise Business Continuity Management: this activity evaluates organization capability of ensuring continuous availability of the Infor-mation Technology infrastructure.
The output of the Assessment phase is reported within the Inherent Risk and the Residual Risk documents.
The Treatment phase implements a platform for taking decision about the afore-mentioned risks. Its work-flow goes through the selection of safeguards, the develop-ment of impledevelop-mentation plan for countermeasures, and the developdevelop-ment of a decision making process. In this phase the organization decides to accept, mitigate, or avoid vulnerabilities and associated risks identified in the previous phase.
Finally, the process of Accreditation defines the steps needed to an organization to obtain the ISSAF certification.
2.3.2. Overview
In this section we will review the Penetration Testing methodology proposed by IS-SAF leaving aside all practical details regarding specific network components assessing methods.
ISSAF penetration testing methodology evaluates organization’s network and systems. It consist of three phases and nine assessment steps. The three phases are: Planning and Preparation; Assessment; Reporting, Clean-up and Destroy Artifacts.
ThePlanning and Preparationphase comprises all the assessment’s preliminary opera-tions. These regards: requirements definition, actions to be performed, and expectaopera-tions. ISSAF requires the identification of a contact person from the target company and those who will perform the tests. During the kick-off meeting both parties have to agree on the specific engagement team, the exact dates, times of the test, escalation path and other arrangements. The meeting has to be concluded with the signing of a formal Assessment Agreement that will provide mutual legal protection.
The Assessment phase is the core of the penetration testing methodology. ISSAF approach is a path that leads the attacker to nine operation steps as shown in Figure 2.7.
Information Gathering: ISSAF defines the information gathering in the broader sense possible. It comprises both technical and non-technical methods. The for-mer looks for DNS records and exploit tools like WHOIS. The latter implements some social engineering methods (e.g. using search engines and mailing lists to find information about the target). The more a pentester knows about his victim the more he will improve his chances to successfully attack it. During this phase, the attacker does not always need to establish a contact with the target. Important information can be collected also from public sources (e.g. the Internet). As secu-rity assessments are generally limited in time and resources, information gathering is important to identify points that will be most likely vulnerable and focus on them.
Network Mapping: this phase directly follows the previous one and refines the information gathering with a more technical approach. The target is to identify what devices are working in the network and outline its topology. Network mapping involves the use of fingerprinting and network analysis tools. More in details, ISSAF specifies different classes of actions to be performed:
– Find live hosts
– Port and service scanning
– Perimeter network mapping (e.g. router, firewalls) – Identifying critical services
– Operating System fingerprinting
– Identifying routes using Management Information Base (MIB) – Service fingerprinting
2.3. ISSAF
At the end of this phase, pentesters will have the possibility to confirm or dismiss some preliminary made hypotheses regarding target systems.
Vulnerability Identification: once specific targets are selected, the attackers will perform several activities to detect actual exploitable vulnerabilities in the system. ISSAF lists some of the activities used to detect weak points:
– Identify vulnerable services using service banners – Perform vulnerability scans
– Perform false positive and false negative verification – Enumerate discovered vulnerabilities
– Estimate probable impact
– Identify attack paths and scenarios for exploitation
Penetration: this phase is the first actual operation against the target. The at-tacker will try to circumvent security measures and gain access to the system under attack. ISSAF methodology emphasizes the following steps within the penetration:
– Find proof of concept code/tools to use for the attack – Develop new tools/scripts if needed
– Test proof of concept code/tools before use them to avoid problems during the actual pentesting
– Use proof of concept code against target
– Verify or disprove the existence of vulnerabilities – Document findings
Gaining Access & Privileges Escalation: in this phase pentesters will confirm the possibility to access the target. To do that the attacker has to compromise all the intermediate systems in the path leading to the physical system under attack (e.g. firewalls, routers, etc.). Moreover, the attacker will usually access the system with the least privileges possible and he will try to obtain administrator’s rights exploiting local vulnerabilities with root-kits.
Enumerating Further: this phase lists malicious activities the attacker can per-form within the compromised system. Such activities are:
– Obtain encrypted passwords for off-line cracking – Obtain passwords by using sniffing or other techniques
2.3. ISSAF
– Sniff traffic and analyze it
– Gather cookies and use them to exploit sessions and for password attacks – E-mail address gathering
– Identifying new routes and networks – Mapping internal networks
Compromise Remote Users/Sites: this phase focuses on the presence of secure communications (e.g. Virtual Private Networks) between remote users/sites and enterprises network. In this scenario, the attackers can try to extend his control to remote users and detached networks. In case of success, the attacker will repeat all the previous steps in the new environment.
Maintaining Access: in this phase pentesters make sure to control the system also in the future. As a vulnerability can always be patched by system operators an attacker can install malicious code in the system to create covert channels and backdoors. Such verification shows the degree of exposure that a system can have once compromised. The Maintaining Access phase is often not performed as part of a penetration test due to the risks involved (e.g. a real attacker capable to take possession of the covert channel and gain access to the system).
Covering Tracks: this phase ensures attackers to be invisible to later analysis. Usually, during a penetration testing, attackers should be as open as possible about their operations (unless the company expressly requests otherwise). There are several actions that pentesters can perform to hide their traces:
– Hide modified files and used tools
– Clear logs by checking history and edit log files – Defeat integrity checking
– Defeat Anti-viruses
– Implement customized Root-kits
(optional)Audit: system audits should be performed after completing a penetra-tion test. This phase can usually bring to light further potential vulnerabilities within the system.
The Reporting, Clean up and Destroy artifacts phase concludes the assessment. Ac-cording to ISSAF a minimal reporting should consist of:
Verbal Reporting: that describes the feedback the company has to have imme-diately after the pentesting. The attackers will give to the company an overview on their finding and they will focus on major critical issues.
Final Reporting: that is an in depth presentation with detailed results of the tests. Each discovered vulnerability has to be presented with related countermea-sures and a proposal on how to implement them. The report should follow a well documented structure. ISSAF proposes a list of information needed for the document:
– Management Summary – Scope of the project – Tools that have been used
– Dates & times of the actual tests on the systems – Outputs of tests performed
– A list of all identified vulnerabilities with included recommendations on how to solve problems found
– A list of action points
Finally, all information that is created and/or stored during the test on target systems should be removed. If this is not possible, such information (e.g. files and directories) should be mentioned to system operators to proceed to their removal.
This concludes the discussion of existing penetration testing methodologies. In a later version, we may extend this to other approaches like OWASP, PTES, or CHECK.
3. Requirements
3.1. Approach
Within the CRISALIS project we merge the expertise of different stakeholders. Academia, SCADA vendors, and SCADA operators are three of the main contributors to our assess-ment methodology for critical infrastructure. However, to further enlarge the information basis that our approach is built upon, we decided to create a questionnaire about testing security infrastructure. The purposes of interviewing CI and security experts are several. There is the need to understand what are the requirements and the expected feedbacks. Moreover, we also need to validate the benefits of establishing a methodology tuned to CIs and also evaluate its comprehensiveness and accuracy. Using a questionnaire seemed the most efficient way to reach a broader community and inquire about the state of the art of CI testing in general and to identify possible steps forward toward the definition of the more comprehensive concept of “CI vulnerability assessment”. The questionnaire is then extended by more detailed interviews with some stakeholders.
The questionnaire is divided into four sections: General information; Security require-ments, design and regulations; Testing and methodologies; and Research.
The General Information section focuses on characteristics of the interviewed per-sons. Information on area of expertise and experience will be used to evaluate the later responses and comments.
The security requirements, design and regulations section collects information about deployed security mechanisms and authentication schemata. Moreover, the section in-vestigates on what limitations there are in deploying security mechanisms in Critical Infrastructures.
The testing and methodologies section is the core part of the questionnaire. It aims to identify currently used security practices focusing on: kind of tests performed, test frequency, and tools used. Furthermore, the section asks about the expectations the experts have from a CI security methodology.
Finally, the Research section concludes the questionnaire asking about CI security research status.
We believe that integration of CRISALIS expertise and questionnaire outputs will pro-vide the necessary background from which to build a comprehensive security assessment
methodology for critical infrastructure. Such a methodology is supposed to take into ac-count already existing best-practice provided by industry and different, but relevant, IT fields. The output has to be compliant with CI experts and operators needs and should allow them to have a common evaluation framework to compare their infrastructures.
Section 3.2 presents the questionnaire we used to collect information. The question-naire is also available online at Critical Infrastructure Security & Penetration Testing [13].
3.2. Questionnaire
Critical Infrastructure Security & Penetration
Testing
Industry/Researcher Questionnaire
University of Twente December 2012
This questionnaire is anonymous unless you want to participate in the follow up men-tioned in section E. The purpose of this work is exploring the need for a standard pene-tration testing methodology for Critical Infrastructure. With the following questions we will collect ideas and requirements useful to realize such methodology.
N.B. If you have never worked with Critical Infrastructure please fulfill sections B and C with your opinion on how these systems are supposed to work and be tested. Questions marked with ‘*’ are intended only for Critical Infrastructures operators/deployers.
A. General information
1. In which area are you working?
# SCADA Vendor
# SCADA End User
A. General information
# Security Consultant
# Academia
# Other: Comments:
2. What is your domain of interest? (Multiple selections are possible)
2 Water
2 Energy
2 Oil & Gas
2 Transportation
2 Other: Comments:
3. Briefly describe your field of work/research and your job.
4. How much experience do you have in working with Critical Infrastruc-tures (CIs)?
(little) 2—2—2—2(a lot) Comments:
5. How much experience do you have in security? (little) 2—2—2—2(a lot)
6. How important is computer security in your domain? (little) 2—2—2—2 (a lot)
Comments:
B. Security requirements, design, and regulations
7. What types of security components do you use/manifacture/deploy? (Multiple selections are possible)
2 Firewall
2 Application White-listing
2 Intrusion Detection System
2 Intrusion Prevention System
2 Forensic Analysis Tool
2 Other: Comments:
8. Rank these limitations to security in order of importance from 1 (High importance) to 6 (Low importance).
Costs are a big factor limiting the amount of research and spending on security.
Time constraints. No time to investigate all potential security issues and solutions.
Off-the-shelf security technology are not suitable for CIs and their proprietary software.
The underlying hardware used in most CI cannot handle modern security features.
Regulations/laws do not allow all security features.
There is no incentive to secure every aspect of CIs.
Other: Comments:
C. Testing and methodologies
9. * How important is the correct functioning of security components in your system?
(little) #—#—#—#(a lot) Comments:
10. * How is the access to the systems’ information and specification con-trolled? (Multiple selections are possible)
2 Only those who have a need-to-know can access such information
2 For some specific aspects a very small set of people has access to such infor-mation
2 Specific area of the company terrain are only accessible by authorized personel
2 Only the manufacters/suppliers have the full specification of the components
2 Open source software is used
2 Also internally, the security tests can mostly be done as black-box tests
2 Other: Comments:
C. Testing and methodologies
11. What kind of security tests are done? (Multiple selections are possible)
2 Stress tests and Denial-of-Service tests
2 Penetration tests
2 Security code review
2 Formale code verification
2 Other: Comments:
12. How often are these tests performed? (Multiple selections are possible)
2 During development
2 During the setup
2 Regularly during operation
2 When some changes occur
2 Other: Comments:
13. Are security tests mainly black-box (system’s specifications unknown) or white-box (full disclosure of information to pentesters)?
# Black-box
# White-Box
# Unknown Comments:
14. * Is it possible to test the system remotely (passing through public networks)?
# Yes
# No
# Unknown Comments:
15. * Is the security testing outsourced, i.e. done by externals?
# No, everything is tested in-house.
# No, generally everything is done in-house but in some cases external security testers are included.
# Yes, but only specific elements are tested by externals.
# Yes, the whole system is tested for security by externals.
# Unknown
# Other: Comments:
C. Testing and methodologies
16. Is the testing performed within company premises?
# Yes, always.
# If outsourced, it can be performed also outside company premises.
# Unknown
# Other: Comments:
17. For web servers security/penetration testing methodologies and tools exist.
Are there any methods/procedures/standards used for testing CI se-curity?
# No, I am not aware of any general procedures/standards.
# No, there are no standards but some methodologies specifically designed for each CI exist.
# Yes, internally a set of documented procedures exist which are followed for each CI.
# Yes, published/standardized methods are used for testing wherever possible.
# Unknown
# Other: Comments:
18. Can you mention some methods used?
19. What are the procedures for security testing CIs? How is the security tested? (Multiple selections are possible)
2 A security test is done when the system is offline.
2 A security test is done on a backup system.
2 Different components are tested separately
2 Unknown
2 Other: Comments:
20. The security tests are done by... (Multiple selections are possible)
2 The team working on the component(s).
2 An internal security team.
2 An external security team.
2 Unknown
2 Other: Comments:
21. Are there any CI-specific tools/software for the testing? (Multiple selections are possible)
2 No, I am not aware of any CI-specific tool.
2 Some simple tools are developed when necessary.
2 Yes, existing software tools are used for testing specific components.
2 Yes, we developed our own tools/software for testing the security of the ICS.
2 Unknown Comments:
22. Referring to previous question, can you mention any of such tools/soft-ware?
C. Testing and methodologies
23. Is there a need for a (standard) CI penetration testing methodology?
# No. There is no incentive to do thorough security testing.
# A little. The methodologies/documented procedures will not be used. We do a flexible security testing strategy that is decided upon for each individual case.
# Yes. General guideline methodologies are good, but there is no need for more specific methodologies.
# Yes. A CI-specific and comprehensive methodology would make security test-ing more organized/ repeatable/ comparable.
# Unknown Comments:
24. This CI penetration testing methodology should... (Multiple selections are possible)
2 include a guideline for testing the whole CI.
2 include a guideline for testing specific components.
2 include general guidelines about each testing aspect.
2 include very detailed information about each testing aspect.
2 suggest software/tools to be used.
2 state how to document the results.
2 Other: Comments:
D. Research
26. Should more research be done in the area of CI security (either at university or at companies)?
# No, there is no incentive to do thorough security testing.
# No, there is already a lot being done (also in non-CI areas that can be applied to CI).
# Yes, more time should be spent in studying CIs’ security.
# Other: Comments:
27. Additional comments:
E. Follow up
28. Would you agree to be contacted for a more in depth discussion?
# Yes
# No
29. If yes, please provide us your email address:
Thanks for your time. In case you provided your email address, we will contact you to organize a 1 hour phone interview.
3.3. Further analyses
At the moment of this writing (April 2013) the distribution and filling out of the ques-tionnaire is still in progress. Due to the limited feedbacks we collected until now, it is
