Introduction to IPsec

IPsec protects IP packets by authenticating the packets, by encrypting the packets, or by doing both. IPsec is performed inside the IP module, well below the application layer. Therefore, an Internet application can take advantage of IPsec while not having to configure itself to use IPsec. When used properly, IPsec is an effective tool in securing network traffic.

IPsec protection involves five main components:

• Security protocols – The IP datagram protection mechanisms. The authentication header (AH) signs IP packets and ensures integrity. The content of the datagram is not encrypted, but the receiver is assured that the packet contents have not been altered. The receiver is also assured that the packets were sent by the sender. The encapsulating security payload (ESP) encrypts IP data, thus obscuring the content during packet transmission. ESP also can ensure data integrity through an authentication algorithm option.

• Security associations database (SADB) – The database that associates a security protocol with an IP destination address and an indexing number. The indexing number is called the security parameter index (SPI). These three elements (the security protocol, the destination address, and the SPI) uniquely identify a legitimate IPsec packet. The database ensures that a protected packet that arrives to the packet destination is recognized by the receiver. The receiver also uses information from the database to decrypt the communication, verify that the packets are unchanged, reassemble the packets, and deliver the packets to their ultimate destination.

• Key management – The generation and distribution of keys for the cryptographic algorithms and for the SPI.

• Security mechanisms – The authentication and encryption algorithms that protect the data in the IP datagrams.

• Security policy database (SPD) – The database that specifies the level of protection to apply to a packet. The SPD filters IP traffic to determine how the packets should be processed. A packet can be discarded. A packet can be passed in the clear. Or, a packet can be protected with IPsec. For outbound packets, the SPD and the SADB determine what level of protection to apply. For inbound packets, the SPD helps to determine if the level of protection on the packet is acceptable. If the packet is protected by IPsec, the SPD is consulted after the packet has been decrypted and has been verified.

When you invoke IPsec, IPsec applies the security mechanisms to IP datagrams that travel to the IP destination address. The receiver uses information in its SADB to verify that the arriving packets are legitimate, and to decrypt them. Applications can invoke IPsec to apply security mechanisms to IP datagrams on a per-socket level as well.

Note that sockets behave differently from ports:

• Per-socket SAs override their corresponding port entry in the SPD.

• Also, if a socket on a port is connected, and IPsec policy is later applied to that port, then traffic that uses that socket is not protected by IPsec.

  Of course, a socket that is opened on a port after IPsec policy is applied to the port is protected by IPsec policy.

IPsec RFCs

The Internet Engineering Task Force (IETF) has published a number of Requests for Comment (RFCs) that describe the security architecture for the IP layer. All RFCs are copyrighted by the Internet Society. For a link to the RFCs, see https://ietf.org/. The following list of RFCs covers the more general IP security references:

• RFC 2411, “IP Security Document Roadmap,” November 1998

• RFC 2401, “Security Architecture for the Internet Protocol,” November 1998

• RFC 2402, “IP Authentication Header,” November 1998

• RFC 2406, “IP Encapsulating Security Payload (ESP),” November 1998

• RFC 2408, “Internet Security Association and Key Management Protocol (ISAKMP),” November 1998

• RFC 2407, “The Internet IP Security Domain of Interpretation for ISAKMP,” November 1998

• RFC 2409, “The Internet Key Exchange (IKE),” November 1998

• RFC 3554, “On the Use of Stream Control Transmission Protocol (SCTP) with IPsec,” July 2003 [ not implemented in the Solaris 10 release ]

IPsec Terminology

The IPsec RFCs define a number of terms that are useful to recognize when implementing IPsec on your systems. The following table lists IPsec terms, provides their commonly used acronyms, and defines each term. For a list of terminology used in key negotiation, see Table 22-1.

Table 19-1 IPsec Terms, Acronyms, and Uses