While tools are an important aspect of any software security testing regimen, there are a number of technologies currently available and under development that can aid organizations and reviewers. Organizations can take advantage of test case generators, which will aid in preparing security test plans for software; test oracles, which provide information about the expected results for specific tests; and other technologies. This section describes a subset of these technologies and how they can be used within a security testing framework.
4.4.1 Tool Integration and Normalization
In 2006, Kris Britton of the National Security Agency (NSA) Center for Assured Software (CAS) observed that the level of integration of software security tools has not reached a point where these tools can support “meta-analysis,” where tools can incorporate the results of one another and interpret, rank and adjust confidence levels appropriately [22]. Most security testing tools cover only a small subset of test patterns or vulnerabilities associated with software. To aid the ability of tools to work together, the OMG Software Assurance Special Interest Group is developing the Software Assurance Ecosystem. The ecosystem is a suite of OMG specifications that can be leveraged together to improve interoperability among tools. These specifications include:
Knowledge Discovery Metamodel (KDM) – This specification provides an ontology that can be used to describe key aspects of software, improving interoperability among tool outputs.
Semantics of Business Vocabulary and Rules– This specification provides the basis for a formal, natural language interpretation of compliance rules, security policies, or other criteria against which the software will be analyzed.
Hatha Systems offers a set of tools, KDM Analytics, that implement these OMG specifications. While these tools are not yet offered as standalone COTS products for testers, Hatha Systems aims to foster the development of these specifications with the goals of increasing interoperability among major tool vendors.
4.4.2 Vulnerability Categorizations and Markup Languages
Over the past few years, a number of vulnerability classification and markup language efforts have emerged. To reign in the effects of competing standards and specifications, NIST launched the Information Security Automation Program (ISAP) and the SCAP. Through ISAP and SCAP, NIST has identified a set of vulnerability classifications and markup languages that form the basis for SCAP-support. Currently, the National Vulnerability Database offers services that comply with many of these standards.
The major SCAP standards are:
Common Vulnerabilities and Exposures (CVE) – CVE provides a standard name and identifier for individual vulnerabilities and exposures that have been publicly identified Common Configuration Enumeration (CCE) – CCE provides a standard name and
identifier for individual configuration issues associated with software components
Common Platform Enumeration (CPE) – CPE provides a standard name and identifier for specific systems, platforms, and packages
Common Vulnerability Scoring System (CVSS) – CVSS provides a metric for quantitatively communicating the impact of a specific vulnerability
Extensible Configuration Checklist Description Format (XCCDF) – XCCDF provides a language for writing security checklists, benchmarks, and related documents
Open Vulnerability and Assessment Language (OVAL) – OVAL provides a language for describing system information, including its current state, as well as the results of a vulnerability assessment.
Where OMG aims to improve tool interoperability through improved analysis of the output of multiple tools, NIST provides organizations with a common set of languages that can serve the basis for future interoperability efforts. While many of these standards may not be directly applicable in an embedded environment, they may be adapted for use. For example, XCCDF may be used to define checklists for embedded Linux distributions while OVAL can be used to describe the results of the assessment of an existing software intensive system. Taking advantage of existing NIST standards may be useful in ensuring that test results can be repeated and that mitigation strategies can be documented for the long period of time many embedded systems are deployed.
4.4.3 Security Analysis/Test Case/Test Scenario Definition
This sub-section focuses on a variety of techniques and methodologies for organizations interested in improving their security posture through improved testing definition. One major current area of research is in attack patterns. Attack patterns are primarily being investigated in government and industry and are commonly used to:
Specify explicit requirements against which software security can be tested
Avoid design and implementation issues that would make the software vulnerable to known attack patterns
Include criteria and checks in design and code reviews, security tests, and post- deployment vulnerability assessments [10].
Attack patterns form the basis for abuse and misuse cases, threat models, and attack trees. However, because attack patterns rely on knowledge of attacks that have occurred in the past, they are a reactive technology. If attackers devise a new attack pattern, software developed and tested against attack patterns would remain vulnerable. Nevertheless, there is consensus that the improved discovery, avoidance, and mitigation of vulnerabilities to known attack patterns will improve the security of software in general. Taking advantage of other testing techniques in tandem with attack patterns will further improve software security.
Academia focuses primarily on attack trees, which are similar to attack patterns in many respects. Attack trees (also known as threat trees or attack graphs) provide a visual representation of possible attacks against the system. In practice, portions of an attack tree may be based on attack patterns as described earlier. Using this visual representation, organizations can better capture information for use in generating abuse and misuse cases, security requirements, and test patterns. In some environments, these attack trees may aid in the risk analysis process as well, aiding assessors in gaining a full understanding of the vulnerabilities within a system.
One of the primary drawbacks of attack trees in their current form is that it is difficult for individuals not skilled in security to effectively use them. Because a tree is generated based on security-relevant conditions within the software, unskilled analysts may generate incomplete trees. Similarly, analysts without a full grasp of the software system may make inadequate assumptions about the system itself.