Instead of providing a direct process for secure software development, criteria for evaluat- ing security have been created in an attempt to standardize security evaluations of software. Often the focus is more on evaluating the process, rather than a technical review of the implementation. When the process is under evaluation, the expectation is that through ensuring a correct process, vulnerabilities can be limited. When the actual implementation is under evaluation, the security claims the developer makes about the product are scruti- nized and reviewed. The important distinction is that the product is not critically evaluated for vulnerabilities, but only evaluated against the claims made about its security. These techniques are discussed outside of the lifecycle because they are intended to evaluate the lifecycle, rather than directly influence any phase.
2.7.1 Criteria for Evaluating Security
The most prominent criteria for evaluating security of a product is the Common Criteria for Information Technology Security Evaluation (CC) [17], capable of evaluating the security of software, firmware, or hardware. CC was developed by North American and European governments in hopes to completely standardized product security evaluations, limiting the
need for vendors to comply with multiple criteria. There are three targeted users of CC: consumers, vendors, and independent evaluators. The goal is to provide consumers with a way to (a) supply vendors security expectations of a product, (b) provide vendors a way to give assurance to customers of security built into the product, and finally (c) provide evaluators a standard process for reviewing how successfully vendors were at incorporating security concerns in the product. Some of the common terms associated with CC:
Targets of evaluation (TOE) - product under evaluation. The product can be software,
firmware, and/or hardware.
Security functional requirements (SFR) - derived from the security objectives of the
TOE. SFRs must be more detailed than the security objectives, completely address- ing the objective but without specifying an implementation.
Security assurance requirements (SAR) - description of how the TOE is to be evalu-
ated.
Consumers of the CC include users of the TOE, government or corporations wishing to define a standard for security, or developers of the TOE. The consumers define protection profiles (PP) which provide details on how TOEs using the PP are implemented. A PP contains security objectives, which are then refined to SFRs or SARs. Each PP contains either SFRs or SARs, but not both. This provides developers more flexibility when selecting a PP to use in their TOE.
Developers are defined as the implementors of a TOE. The developers create a security target (ST) for their TOE, which is similar to a requirements specification document. A ST contains a set of SFRs and SARs that must be followed by the TOE, which mirrors the requirements to implementation process already in place when using a classic lifecycle. The SFRs and SARs in the document comes from either PP(s) defined by the consumers, or are newly devised requirements by the development team. Should the requirements from a PP be selected for inclusion into a ST, then all the requirements listed in the PP must be included without exception. This will guarantee that the concerns of the customer(s) that created the PP will be implemented.
Evaluators use the ST provided by the developer, to determine if the security objectives of the developer are met. The exact process used by the evaluators depends on the evaluator assurance level (EAL), where higher levels or assurance requires more evaluation. The eval- uation does not critically review the security through penetration testing or other security analysis, but only validates the requirements in the ST. If a requirement is needed for correct implementation or design of the TOE, but it is not present in the ST, then the TOE can pass the evaluation even if the missing requirement caused a vulnerability.
Security evaluation criteria, and CC in particular, come very close to matching the designed function of the software security requirements and the SSRT. However, security evaluation criteria only validate the claims the developer made about the system, and do
not critically evaluate the actual security of a system as implemented. CC mitigates this issue by providing standard requirements through the use of PPs. Nonetheless, a PP must be made by the industry or customer(s) for maximum effectiveness. Furthermore, the focus of the requirements in the CC are on information security requirements, where the SSRT focuses on software security requirements. CC also has other weaknesses including high costs of evaluation, and the inability to evolve with emerging threats due to bureaucracy [67].
Similar approaches to CC include CESG Claims Tested Mark (CCT MARK) [25], Trusted Computer System Evaluation Criteria (TCSEC) [54], and Information Technology Security Evaluation Criteria (ITSEC) [34]. The common issue with all criteria for evaluating security is that they test the vendor’s security claims on the product, instead of critically evaluating the actual security of the product.
2.7.2 Development Process Evaluations for Security
The security of a system is highly dependent on the process used for development; if a strict process is not followed there is higher chance of failure being introduced. While a good process does not guarantee better results, it reduces the risk factor. In terms of security, the process for development can influence the vulnerabilities present in the system. For example, even if a full set of security requirements are provided, if a process does not use those requirements correctly, there is a high chance that a vulnerability will be introduced during design or implementation.
The need for process evaluation has led to System Security Engineering-Capability Ma- turity Model (SSE-CMM) [76], the most widely used process evaluation for security. The goal of SSE-CMM is to provide a model process that will be used to evaluate real-world in- dustry processes for system development. A process that has been granted a high capability level does not guarantee a vulnerability free system as a result, but instead guarantees that a better process for security is followed. The SSE-CMM can evaluate the classic lifecycle used by an organization, even if the software security requirements and the SSRT are used.