System Testing

In A Practical Guide to Security Engineering and Information Assurance, [75]

Debra Herrmann recommends that because attackers are not biased by knowledge of a systems design or security protection mechanisms, testing of the integrated system by the system’s engineers be augmented by independent testing by a disinterested third party.

Tests to discover design defects are difficult to develop. Like the systems engineers developing security designs, the testing group (whether independent or not), will be able to construct test cases based on understanding the

psychology of the attackers and knowledge of typical software, hardware, and other system fault types. Additional sources of information for development of test cases and scripts include—

u Misuse and abuse cases

u Threat tree analysis reports

u Threat models

u FMEA reports

u Security policy models

u Security targets

u System security requirements.

At a minimum, testing the resiliency of a system design to attack would include—

u Testing for transient faults, such as an unusual combination or sequence of events, degradation of the operating environment (temporary saturation of the network, power losses, environmental changes), or induced temporary loss of synchronization among components of a system

u Testing for the ability of the system to withstand password guessing, masquerading, etc.

u Creative “what if” testing.

Section 4.5 describes a number of security testing techniques that can be applied to software. Some of these techniques are also useful, and in the case of penetration testing, best performed at the system level.

Software Security Assurance State-of-the-Art Report (SOAR) 6 Context.9

Section 4 Secure Systems Engineering

4.6 SSE-CMM

Note: This section will be of the most use to readers already familiar with the Systems Engineering (SE) CMM.

The SSE-CMM process reference model augments project and

organizational process areas from the SE CMM with security engineering process areas for improving and assessing the maturity of the security engineering processes used to produce information security products, trusted systems, and security capabilities in information systems. The scope of the processes addressed by the SSE-CMM encompasses all activities of the system security engineering life cycle, including concept definition, requirements analysis, design, development, integration, installation, operation, maintenance, and decommissioning. The SSE-CMM includes requirements for product developers, secure systems developers and integrators, and organizations that provide computer security services and/or computer security engineering, including organizations in the commercial, government, and academic realms.

The SSE-CMM is predicated on the view that security is pervasive across all engineering disciplines (e.g., systems, software, and hardware), and the Common Feature coordinate security practices has been defined to address the integration of security with all disciplines and groups involved on a project or within an organization (see Table 4-1). Similarly, the Process Area (PA) coordinate security defines the objectives and mechanisms to be used in coordinating the security engineering activities with all other engineering activities and teams.

Table 4-1. SSE-CMM Security Engineering Process Areas and Goals Security Engineering PA PA Goals

Administer security controls Ensure that security controls are properly configured and used.

Assess impact Reach an understanding of the security risk associated with operating the system within a defined environment.

Assess security risk Identify system vulnerabilities and determine their potential for exploitation.

Assess threat Reach an understanding of threats to the security of the system.

Assess vulnerability Reach an understanding of the system’s security vulnerabilities.

Build assurance argument Ensure that the work artifacts and processes clearly provide the evidence that the customer’s security needs have been met.

Coordinate security Ensure that all members of the project team are aware of and involved with security engineering activities to the extent necessary to perform their functions; coordinate and communicate all decisions and recommendations related to security.

Software Security Assurance State-of-the-Art Report (SOAR)

70

Section 4 Secure Systems Engineering

Table 4-1. SSE-CMM Security Engineering Process Areas and Goals - continued Security Engineering PA PA Goals

Monitor security posture Detect and track internal and external security-related events;

respond to incidents in accordance with policy; identify and handle changes to the operational security posture in accordance with security objectives.

Provide security input Review all system issues for security implications and resolve those issues in accordance with security goals; ensure that all members of the project team understand security so they can perform their functions; ensure that the solution reflects the provided security input.

Specify security needs All applicable parties, including the customer, reach a common understanding of security needs.

Verify and validate

security Ensure that the solutions satisfy all of their security requirements and meet the customer’s operational security needs.

The SSE-CMM and the method for applying the model (i.e., the appraisal method) are intended to be used as a—

u Tool that enables engineering organizations to evaluate their security engineering practices and define improvements to them

u Method by which security engineering evaluation organizations, such as certifiers and evaluators, can establish confidence in the organizational capability as one input to system or product security assurance

u Standard mechanism for customers to evaluate a provider’s security engineering capability.

As long as the users of the SSE-CMM model and appraisal methods thoroughly understand their proper application and inherent limitations, the appraisal techniques can be used in applying the model for self-improvement and in selecting suppliers.

An alternative approach to a secure CMM is described in the Federal Aviation Administration (FAA)/Department of Defense (DoD) Proposed Safety and Security Extensions to iCMM and CMMI (see Appendix D).

For Further Reading

Mary Schanken, Charles G. Menk III, James P. Craft, (NSA and United States Agency for International Development), US Government Use of the Systems Security Engineering Capability Maturity Model, (SSE-CMM), (presentation at the National Information Systems Security Conference, October 19, 1999).

Available from: http://csrc.nist.gov/nissc/1999/program/act10.htm

Software Security Assurance State-of-the-Art Report (SOAR) 71

Section 4 Secure Systems Engineering