IP Security Protocols (IPSec)

IPSec protocols can supply access control, authentication, data integrity, and confidentiality for each IP packet between two participating network nodes. IPSec can be used between two hosts (including clients), a gateway and a host, or two gateways. No modification of network hardware or soft- ware is required to route IPSec. Applications and upper level protocols can be used unchanged.

IPSec adds two security protocols to IP, Authentication Header (AH) and Encapsulating Security Payload (ESP). AH provides connectionless

integrity, data origin authentication, and anti-replay service for the IP packet. AH does not encrypt the data, but any modification of the data would be detected. ESP provides confidentiality through the encryption of the payload. Access control is provided through the use and management of keys to control participation in traffic flows.

IPSec was designed to be flexible, so different security needs could be accommodated. The security services can be tailored to the particular needs of each connection by using AH or ESP separately for their indi- vidual functions, or combining the protocols to provide the full range of protection offered by IPSec. Multiple cryptographic algorithms are sup- ported. The algorithms that must be present in any implementation of IPSec follow. The null algorithms provide no protection, but are used for consistent negotiation by the protocols. AH and ESP cannot both be null at the same time.

■ _{DES in CBC (Cipher Block Chaining) mode}

■ _{HMAC (Hash Message Authentication Code) with MD5} ■ _{HMAC with SHA}

■ _{Null Authentication Algorithm} ■ _{Null Encryption Algorithm}

A Security Association (SA) forms an agreement between two systems participating in an IPSec connection. An SA represents a simplex connec- tion to provide a security service using a selected policy and keys, between two nodes. A Security Parameter Index (SPI), an IP destination address, and a protocol identifier are used to identify a particular SA. The SPI is an

arbitrary 32-bit value selected by the destination system that uniquely identifies a particular Security Association among several associations that may exist on a particular node. The protocol identifier can indicate either AH or ESP, but not both. Separate SAs are created for each protocol, and for each direction between systems. If two systems were using AH and ESP in both directions, then they would form four SAs.

Each protocol supports a transport mode and a tunnel mode of opera- tion. The transport mode is between two hosts. These hosts are the end- points for the cryptographic functions being used. Tunnel mode is an IP tunnel, and is used whenever either end of the SA is a security gateway. A security gateway is an intermediate system, such as a router or firewall, that implements IPSec protocols. A Security Association between a host and a security gateway must use tunnel mode. If the connection traffic is destined for the gateway itself, such as management traffic, then the gateway is treated as a host, because it is the endpoint of the communica- tion.

In transport mode, the AH or ESP header are inserted after the IP header, but before any upper layer protocol headers. As shown in Figure 1.11, AH authenticates the original IP header. AH does not protect the fields that are modified in the course of routing IP packets. ESP protects only what comes after the ESP header. If the security policy between two nodes requires a combination of security services, the AH header appears first after the IP header, followed by the ESP header. This combination of Security Associations is called an SA bundle.

Figure 1.11

Data Data TCP Header TCP Header Original IP Header AH Original IP Header Data Data TCP Header TCP Header Original IP

Header HeaderESP Original IP

Header

ESP Trailer ESPAuth

Before Applying ESP

After Applying ESP Before Applying AH

After Applying AH

Authenticated

Authenticated Encrypted

In tunnel mode, the original IP header and payload are encapsulated by the IPSec protocols. A new IP header that specifies the IPSec tunnel desti- nation is prepended to the packet. The original IP header and its payload are protected by the AH or ESP headers. From Figure 1.12 you can see that, as in transport mode, AH offers some protection for the entire packet. AH does not protect the fields that are modified in the course of routing IP packets between the IPSec tunnel endpoints, but it does completely protect the original IP header.

Key management is another major component of IPSec. Manual tech- niques are allowed in the IPSec standard, and might be acceptable for con- figuring one or two gateways, but typing in keys and data are not practical in most environments. The Internet Key Exchange (IKE) provides auto- mated, bi-directional SA management, key generation, and key manage- ment. IKE negotiates in two phases. Phase 1 negotiates a secure,

authenticated channel over which the two systems can communicate for further negotiations. They agree on the encryption algorithm, hash algo- rithm, authentication method, and Diffie-Hellman group to exchange keys and information. A single phase 1 association can be used for multiple phase 2 negotiations. Phase 2 negotiates the services that define the SAs used by IPSec. They agree on IPSec protocol, hash algorithm, and encryp- tion algorithm. Multiple SAs will result from phase 2 negotiations. An SA is created for inbound and outbound of each protocol used.

Figure 1.12

Data New IP

Hdr TrailerESP ESPAuth

Encapsulating Security Payload Authentication Header Authenticated Authenticated Encrypted ESP _{IP Hdr}Orig TCP New IP Hdr AH IP HdrOrig TCP Data