Security Technology and Tools

In this section, we look at the security technology and tools available to design security architectures. It follows a similar format to the Information Protection Environment section by separating the tools into three major protection categories: platform, mainframe, and network.

Protection mechanisms are used to ensure the separation between objects in a system. Two categories of protection mechanisms are active or passive. An active protection mechanism prevents access to an object if, according to the protection mechanism, the access is not authorized. For example, an active protection mechanism is when memory protection is provided by the hardware architecture, where access to objects at certain addresses may be controlled depending on such criteria as the current processor state and attributes of the process attempting to reference the object in question.

A passive protection mechanism is one that prevents or detects unau- thorized use of the information associated with an object, even if access to the object itself is not prevented. In most cases, these techniques use cryp- tographic techniques to prevent unauthorized disclosure of information or checksum techniques to detect an unauthorized alteration of an object.

Protection mechanisms are often implemented in operating systems, hardware, or firmware. They control access to primitive objects such as memory, machine code instructions, and registers, which are used to con- trol input/output operations.

Platform Protection

In the environment section, the topics discussed were the basics of how the platform components operate and some of the vulnerabilities and threats associated with the platform environment. In this topic, we look at several options that can be implemented to secure the operating system, memory, files, and objects.

Operating System Protection

Three of the security technologies used by operating systems to protect security features of the software are the trusted computing base (TCB), the reference monitor, and the security kernel.

Trusted Computing Base (TCB). The trusted computing base (TCB) is the totality of protection mechanisms within a computer system — including hardware, firmware, software, processes, and some inter-process commu- nications — that, combined, are responsible for enforcing a security pol- icy. The TCB consists of one or more components that together enforce a unified security policy over a product or system. The ability of the TCB to correctly enforce a security policy depends solely on the mechanisms within the TCB and on the correct input by system administrative person- nel of parameters (e.g., a user’s clearance) related to the security policy (e.g., TCSEC). While some operating systems implement a trusted com- puter base, the system could also be operated in such a manner as to negate much of the value derived from the TCB. Similarly, many operating systems do not implement a trusted computing base at all. If imple- mented, it usually follows the reference monitor concept, which is explained below.

The system architecture of trusted computing includes the ability of the TCB to protect itself from untrusted processes. For example, the TCSEC C2 system architecture criterion, described in the Assurance, Trust, and Con- fidence Mechanisms section of this chapter, includes three TCB require- ments:

• The TCB must maintain for its own execution a domain protecting it from external interference and tampering.

• Resources controlled by the TCB may be a defined subset of subjects and objects.

• The TCB must isolate the resources to be protected so they are subject to access control and auditing.

The TCB maintains the confidentiality and integrity of each domain and monitors four basic functions:

• Process activation. In a multiprogramming environment, activation and deactivation of processes create the need for a complete change of registers, file access lists, process status information, and other pointers, much of which is security-sensitive information.

• Execution domain switching. Processes running in one domain often invoke processes in other domains to obtain more sensitive data or services. The TCB ensures that the processes provide security.

• Memory protection. Because each domain includes code and data stored in memory, the TCB must monitor memory references to ensure confidentiality and integrity for each domain.

• I/O operations. In some systems, software is involved in the transfer of characters to the I/O operation. This process connects a user program in the outermost domain with the I/O operation in the innermost domain. This cross-domain connection must be monitored.

A trusted system is one having undergone sufficient benchmark test- ing and validation to achieve the user’s requirements and those estab- lished in the evaluation criteria. These requirements include those for reliability, security, and operational effectiveness with specified perfor- mance characteristics.

Trust can be measured by ensuring that the (1) System Security Policy states the rules enforced by a system’s security features, and (2) System Assurance Policy states the trust placed in a system and the methodology used to develop, test, document, and deliver the system to the user in a trusted manner.

Reference Monitor. The reference monitor is an access control concept referring to an abstract machine that mediates all accesses to objects by subjects based on information in an access control database. The refer- ence monitor must mediate all access, be protected from modification, be verifiable as correct, and must always be invoked. The reference monitor, in accordance with the security policy, controls the checks that are made in the access control database.

Security Kernel. The security kernel is the hardware, firmware, and soft- ware elements of a trusted computing base (TCB) implementing the refer- ence monitor concept. The security kernel must satisfy three principles:

• It must mediate all accesses (completeness). • It must be protected from modification (isolation). • It must be verifiable as correct (verifiable).

Historical Perspective. In early days of computing, the term “monitor” was used to identify the program in the system that controlled the actions of other programs. As monitors became bigger, they began to be called “oper- ating systems.” The term “monitor” began to mean only the most primitive types of operating systems. The term “reference monitor” was coined to refer to a special-purpose monitor dealing only with access control to resources.

The security kernel must be implemented to prevent users from modify- ing the operating system. If a user were able to disable or circumvent the protection mechanisms by modifying the operating system, the integrity of the system could be compromised.

As shown in Exhibit 11, the reference monitor is an abstract concept, the security kernel is the implementation of the reference monitor, and the TCB contains the security kernel along with other protection mechanisms.

Security Perimeter. The term “security perimeter” is sometimes used to refer to the security kernel, as well as other security-related system func- tions, being within the (imaginary) boundary of the Trusted Computing Base (i.e., the security perimeter). This is a different definition than tradi- tionally associated with the term. In this case, the security perimeter refers to the boundaries surrounding the security kernel and TCB. System ele- ments outside the security perimeter are not to be trusted.

Layering. A primary concept of operating system design is to place the security mechanisms in one of the lower system layers. An example of this is shown in Exhibit 12. A layered operating system looks like a stack of sys- tems, each having an interface for use by the layers above. Layers in the system are hierarchical, wherein the lower layers provide primitive func- tions and the upper layers provide more complex functions. Remember that a vulnerability may exist if an unauthorized user could get access to a layer below and thus compromise the layer above. Additional protection for the data can be achieved through layering above the operating system in the applications the system is executing for the user community.

One technique in the layering approach is called data hiding — when the software in each layer maintains its own global data and does not directly reference data outside its layers. Having a single security layer is not pos- sible because each layer has its own set of objects requiring secure Exhibit 11. Security Kernel

Subjects Reference Objects

Monitor (Policy)

Security Kernel Database

Audit File

management. Thus, each layer must be able to make security decisions. For example, if one layer is able to control all security decisions, that par- ticular layer would need to know too many details about the other layers, thus violating the data-hiding principles of layering. If an entire operating system existed in each layer, with no pass-through features, such knowl- edge would not be required because each layer would be capable of pro- tecting all of its objects. The use of a layered architecture can help increase the reliability of a secure operating system.

TOC/TOU Protection. The safest solution for TOC/TOU is for the operat- ing system to copy the parameters to a location safe from asynchronous modification prior to validation. It is also necessary for the operating sys- tem to prevent an asynchronous change to all information on which the validation depends, not just to the parameters themselves. For example, if at a higher level a system call is invoked to read a file, the operating system must first read a directory to find out the caller’s mode of access and to obtain the location of the file on the storage disk. During the time that the file on the disk is read, the file may have been deleted and a new file put in its place, which may be a file to which the caller did not originally have access. To control this, operating systems can put appropriate locks on files.

While locking can help avoid the TOC/TOU problem in situations where tables and operating systems databases are involved in checking access, a user’s application database randomly located somewhere in a process’s address space may not be subject to the same controls. Exhibit 12. Layering

Applications, User Processes

Compilers, Database Managers

Utility Functions, Systems, Device Allocations, Scheduling, Memory Management Synchronization, Allocation Security Features Hardware Operating System O/S Kernel Security Kernel

However, applications themselves typically provide similar mechanisms to protect the data and enforce access privileges.

Guard Protection. A logical security guard is (usually) an add-on security feature addressing a particular class of multi-level security problems. In some environments, users may need to communicate with a system although some of those users cannot be allowed direct access to the sys- tem because of weaknesses in the system’s security controls.

For example, users may need access to a restricted set of data, and the system may not be strong enough to protect other highly sensitive data residing on that system. A guard can permit users logged onto a system running at a low access class to submit queries to a database running on a system at a high access class. The two systems are not allowed to commu- nicate directly because neither system can be trusted. The guard is a trusted system authorized to communicate with both systems at the same time. Essentially, the guard is trusted to prevent a nonsecure flow of infor- mation from high to low.

Automated guards have been used for one-way traffic, such as sending mail from a low to a high system. The only information the low system needs to receive in response to a message is an acknowledgment that the guard has accepted the message. The guard has enough store-and-forward capability to ensure that little or no information about the status of the high system can be deduced by the low system. This eliminates the possi- bility of information loss or inference through a covert channel. Although the guard is an add-on security feature, it eliminates the need for a person to read a message on one terminal in the low system and then retype the message into another terminal on the high system.

Process Isolation. A basic role implemented in the architecture is the capability of the security system to protect itself. Process isolation involves the use of distinct address spaces and ensures that multiple pro- cesses running concurrently will not interfere with each other, either by accident or by design. It ensures that the system can keep track of every- thing (e.g., registers or status information) it needs to when switching from one process to another. On modern operating systems, this is typically a basic function. Built into the operating system are memory and process management facilities to prevent corruption of the applications, data, and operating system parameters. The operating system must abort or handle the request to prevent one process from executing instructions accessing another process’s address space.

Enforcement of Least Privilege. Just as with least privilege assignments for a person, processes must have no more privilege than needed to per- form functions. Only those modules needing complete system privileges

are located in the security kernel, and other modules call on more privi- leged routines only as needed and as long as needed.

Hardening. Traditionally, the term “hardening” of any operating system referred to the protection of kernel (layers in the operating system), hard- ware, and memory from programs already executing on the host. Recently, it has been expanded to include the notion of strengthening network inter- faces and limiting the activities and services permitted upon them. Hard- ening must include all aspects of the system, including physical and logical access, the operating systems, applications, and data interfaces.

Some hardware vendors are now providing hardened versions of their systems that may include a sandbox layer of access controls that separate resources from access requests. For example, one vendor requires each resource access from an application to use a security interface to reference an operating system component. If an access decision is needed before permitting the request to proceed, a logical security manager is invoked for resolution of the request.

General Operating System Protection. The following items are typical characteristics included in the design of a trusted operating system and associated processes that provide both the features (functions) and assur- ance necessary to ensure implementation of a trusted system:

• User identification and authentication: must authenticate or verify users to the system.

• Mandatory access control: access control policy decisions beyond the control of the object owner.

• Discretionary access control: all object access control left to the owner.

• Complete mediation: control all accesses; it is difficult as more paths are created.

• Object reuse protection: ensure sensitive information is purged from memory and magnetic media before assignment to a new subject. • Audit: be able to generate security-relevant events.

• Protection of audit logs: ensure they cannot be modified after cre- ation.

• Audit log reduction: since audit logs can become large, it is helpful to have some type of volume and analysis tools that can be used to simplify and/or correlate the security events (i.e., implement a clip- ping level).

• Trusted path: provide protection of critical operations (i.e., the path between a workstation and authentication server).

• Intrusion detection: provide capabilities for recognizing an intrusion event and invoking an analysis capability.

Memory Protection

One of the most basic hardware requirements for a secure system is memory protection. If a system runs a single process in memory at one time, on behalf of a single user, memory protection is needed to prevent the user’s programs from affecting the operating system. If it is a multitask- ing system, memory protection will need to isolate the process’s memory areas from each other.

Hardware techniques were developed to provide the necessary protec- tion mechanisms in the computer hardware, and the concept of a “privi- leged” state of operation was introduced. This privileged state ensured that only the operating system could perform the operations that were crit- ical to controlling and maintaining the protection mechanisms.

From a historical perspective, the security for RAM began to evolve with the move from mainframes to personal computers that ran MS-DOS oper- ating systems. The MS-DOS operating system was developed to run on a microprocessor and did not provide any protection features. Protection was not deemed necessary because microprocessors and memory avail- able were very limited, so it was not common to allow any sharing. Addi- tionally, the design of the operating system allowed the execution of only a small number of programs with no multitasking capability. For example, while MS-DOS has the concept of Terminate and Stay Resident (TSR) pro- grams, those programs were only resident in memory — they did not exe- cute until an instruction from the user activated them. Because MS-DOS did not control the coexistence of programs in its memory, and because it did not provide protection mechanisms to protect files stored on disk, the computers running MS-DOS were vulnerable to programs that tried to interfere with other programs or files.

Even in the evolution of memory management, the underlying require- ment for security in operating systems is still concerned with the ability of the operating system to enforce control over the storage and movement of information in and between the resources that the operating system sup- ports. Because most systems today are multi-user systems requiring the ability to share resources, the operating system must ensure that the only way programs (acting on behalf of users) can interact with shared resources is through controls enforced by the operating system. To achieve this, various controls must be built into the operating system.

Typical memory protections are:

• Every reference is checked for protection.

• Many different data classes can be assigned different levels of protection.

• Two or more users can share access to the same segment with potentially different access rights.

• Users cannot access a memory or address segment outside what has been allocated for them.

Hardware segmentation is a hardware protection feature dividing virtual memory into segments. It ensures that a process occupies only those seg- ments needed during execution. The operating system manages what memory is allocated for a process. If the process attempts to access mem- ory or alter memory contents outside its allocated resource, the operating system issues a message to the application. This frequently results in abnormal termination of the application.

Isolation. A system’s isolation capabilities determine its overall security robustness. There are two types: hardware isolation and software isola- tion. Hardware isolation involves the isolation of the TCB from untrusted parts of the system. Software isolation is the containment of subjects and