IPsec Negotiation Using the IKE Protocol

IKE negotiates IPsec tunnels between two IPsec peers. This negotiation can be done using a combination of main-mode and quick-mode exchanges or a combination of aggressive-mode and quick-mode exchanges. This section looks at the various packets and message types that are used in these exchanges to do the negotiation. We will look at three types of negotiations that IKE carries out:

  • Main mode using preshared key authentication followed by quick-mode negotiation
  • Main mode using digital signature authentication followed by quick-mode negotiation
  • Aggressive mode using preshared key authentication followed by quick-mode negotiation

In addition to these types, the following types of negotiations can also take place:

  • Main mode using encrypted nonces authentication followed by quick-mode negotiation
  • Aggressive mode using digital signature authentication followed by quick-mode negotiation

However, we will not go into the details of the last two types because the first three are by far the most common permutations used. We will focus on the most commonly used types of negotiations to aid your understanding.

Main Mode Using Preshared Key Authentication Followed by Quick Mode Negotiation

As stated earlier, this negotiation takes place with the exchange of a total of six messages between the two IPsec peers in main mode followed by three messages exchanged during quick mode.

In the following sections we will walk through the preparation for, as well as the actual exchange of, each message involved in this negotiation.

IKE Phase 1 (Main Mode): Preparation for Sending Messages 1 and 2

The first two messages in the IKE main mode negotiation are used to negotiate the various values, hash mechanisms, and encryption mechanisms to use for the later half of the IKE negotiations. In addition, information is exchanged that will be used to generate keying material for the two peers.

However, before the first two messages can be exchanged, the negotiation’s initiator and responder must calculate what are known as cookies. These cookies are used as unique identifiers of a negotiation exchange.

The two peers generate a pseudo-random number that is used for anti clogging purposes. These cookies are based on a unique identifier for each peer (src and destination IP addresses) and therefore protect against replay attacks. The ISAKMP RFC states that the method of creating the cookie is implementation-dependent but suggests performing a hash of the IP source and destination address, the UDP source and destination ports, a locally generated random value,time, and date. The cookie becomes a unique identifier for the rest of the messages that are exchanged in IKE negotiation.The following list shows how each peer generates its cookie:

  • Generation of the initiator cookie— An 8-byte pseudo-random number used for anti-clogging

CKY-I = md5{(src_ip, dest_ip), random number, time, and date}

  • Generation of the responder cookie— An 8-byte pseudo-random number used for anti-clogging

CKY-R = md5{(src_ip, dest_ip), random number, time, and date}

IKE uses payloads and packet formats defined in the ISAKMP protocol to do the actual exchange of information. The packets exchanged consist of the ISAKMP header and a series of payloads that are used to carry the information needed to carry out the negotiation.IKE Phase 1 (Main Mode): Sending Message 1 Let’s look at the first message sent across by the IPsec initiator to the respondent and look at the various fields in this message.Figure 13-5 shows the first IKE message being sent.

Figure 13-5. Sending IKE Main Mode Message 1

IPsec Sound on Air

IPsec Phase-1 Message 1

Figure 13-5 shows the ISAKMP header and five payloads. There is one SA payload and two pairs of proposal and transform payloads.

NOTE

All payloads have a field that defines the length of that particular payload. In all the figures in this chapter depicting packet formats, the name of the payload in this field is highlighted to clearly distinguish between the various payloads.

Let’s look at each of these in turn :

  • ISAKMP header— The ISAKMP header contains the initiator’s cookie, and the responder’s cookie is left at 0 for the responder to calculate and fill in. The next payload field contains a value signifying that the next payload is the SA payload.
  • SA payload— The SA payload contains two important pieces of information. Because ISAKMP is a generic protocol with packet and message formats that can be used to negotiate any number of protocols, it is important to specify that this particular ISAKMP exchange is taking place for IPsec negotiation. Therefore, the SA payload contains a Domain of Interpretation (DOI), which says that this message exchange is for IPsec. The other important piece of information is the situation. The situation definition is a 32-bit bit mask that represents the environment under which the IPsec SA proposal and negotiation are carried out. The situation provides information that the responder can use to make a policy determination about how to process the incoming security association request.
  • Proposal payload— The proposal payload contains a proposal number, protocol ID, SPI (security parameter index) size, number of transforms, and SPI. The proposal number is used to differentiate the various proposals being sent in the same ISAKMP packet. The protocol ID is set to ISAKMP, and the SPI is set to 0. The SPI is not used to identify an IKE phase 1 exchange. Rather, the pair of cookies is used to identify the IKE exchange. We will discuss SPI again during phase 2 of IKE where its use is of more significance. The number of transforms indicates the number of transforms that are associated with this particular proposal payload (only one in this case). Note that the packet contains two pairs of proposal and transform payloads.
  • Transform payload— Includes transform number, transform ID, and IKE SA attributes.The transform number and the ID are used to uniquely identify the transform from among the rest of the transforms offered in this ISAKMP packet (this sample packet has only one other transform, though). The IKE SA attributes include the attributes that the initiator wants the responder to agree on. These include the type of hash to use in the calculation of various keying materials later in the negotiation, the encryption mechanism to use to encrypt IKE negotiation messages as soon as a shared secret has been established, the Diffie-Hellman exchange mechanism to use, the method of authentication to use, and thetimeout values of the IKE SAs negotiated. In this example, assume that the method of authentication being offered is preshared, meaning that a preshared key has been defined on the initiator and responder out of band.

For the sake of this example, assume that the initiator offered the responder the following transform attributes in the first payload. Don’t worry about what it offered in the second payload, because we will assume that the responder agrees to what is offered in the first pair of transform and proposal payloads and does not have to worry about the second pair:

  • Encryption mechanism— DES
  • Hashing mechanism— MD5-HMAC
  • Diffie-Hellman group— 1
  • Authentication mechanism- Preshared

All ISAKMP messages are carried in a UDP packet with destination port 500. IKE Phase 1 (Main Mode): Sending Message 2 Message 2 is the response from the responder to the packet that was sent by the initiator. Most of the fields are the same as in the packet sent by the initiator, so we will discuss the differences only. Note that only one proposal and transform payload is included in the response because the responder agrees to only one proposal/transform pair and returns that as the agreed-on pair.

Figure 13-6 shows the second IKE message being sent.

Figure 13-6. Sending IKE Main Mode Message 2

Phase1-Message-2

Phase1-Message-2

Here are the payloads included in this message:

  • ISAKMP header— You can see that the ISAKMP header now has both cookie fields set to their respective values.
  • SA payload— The SA payload contains pretty much the same material that was sent by the initiator.
  • Proposal payload— The proposal payload contains the information for the proposal thatthe responder has decided to accept.
  • Transform payload— The transform payload contains the elements of the transform thatthe responder has decided to accept.

For the sake of our example, assume that the responder accepted the first proposal described inthe preceding section:

  • Encryption mechanism— DES
  • Hashing mechanism— MD5-HMAC
  • Diffie-Hellman group— 1
  • Authentication mechanism— Preshared

NOTE

IKE SA timeouts are also negotiated using the transform payloads. The acceptabletransform payload must not only have all the other IKE attributes contained in it thatare agreeable to the responder, but it also must have an IKE SA timeout value that isequal to or less than the responder that it is set up to accept. If no such transform isincluded, the negotiation fails.

IKE Phase 1 (Main Mode): Preparation for Sending Messages 3 and 4

The next step that both the initiator and the responder must carry out is to generate material thatwill be used for the production of a Diffie-Hellman shared secret between the two. Because DH isa critical component of the negotiation taking place between the two peers, we will here discusshow it works.

Diffie-Hellman Algorithm

The Diffie-Hellman algorithm is used in IKE negotiations to allow the two peers to agree on a shared secret, to generate keying material for subsequent use, without knowing any secrets beforehand. (Note that although the preshared secret in this example is already defined on the two peers, the DH secret is used in conjunction with that preshared secret to authenticate the two peers to each other.)

The DH algorithm relies on the following property:

There exists a DH public value = Xa

such that

Xa= gmod p

where

g is the generator

p is a large prime number

a is a private secret known only to the initiator

And there exists another DH public value = Xb

such that

Xb= gbmod p

where

g is the generator

p is a large prime number

b is a private secret known only to the responder

Then the initiator and the responder can generate a shared secret known only to the two of them by simply exchanging the values Xa and Xb with each other. This is true because

initiator secret = (Xb)a mod p = (Xa)mod p = responder secret

This value is the shared secret between the two parties and is also equal to gab.

Coming back to IKE, in order to calculate the DH secret between the two peers, the two peers calculate the DH public values and send them to each other. In addition, a value known as a nonce is also generated and exchanged. A nonce is a very large random number generated using certain mathematical techniques. It is used in later calculations of the keying material. The following lists describe the preparation for sending message 3 of the IKE.

First, the two peers independently generate a DH public value:

  • Generation of the DH public value by the initiator

DH public value = Xa

Xa= gmod p where

g is the generator

p is a large prime number

a is a private secret known only to the initiator

  • Generation of the DH public value by the responder DH public value = Xb

Xb= gbmod p

where

g is the generator

p is a large prime number

b is a private secret known only to the responder

As soon as the DH public values have been calculated, the two peers also independently calculate the nonces:

  • Generation of a nonce by the initiator

initiator nonce = Ni

  • Generation of a nonce by the responder

responder nonce = Nr

IKE Phase 1 (Main Mode): Sending Message 3

The third message is sent from the initiator to the responder. The ISAKMP header is similar to the one contained in the earlier messages, complete with cookies. However, two new payloads are introduced in this message—the key exchange payload and the nonce payload.

  • Key exchange (KE) payload— The KE payload is used to carry the DH public value Xa generated for doing DH.
  • Nonce payload— The nonce payload contains the nonce generated in accordance with the preceding discussion.

Figure 13-7 shows message 3 of IKE being sent.

Figure 13-7. Sending Message 3 of IKE

IKE Phase 1 (Main Mode): Sending Message 4

IPSec Phase1- Message-3

IPSec Phase1- Message-3

The fourth message is sent from the responder to the initiator. It is very similar to message 3 in that it contains the corresponding DH public value and the nonce for the responder.

Figure 13-8 shows message 4 of IKE being sent.

Figure 13-8. Sending Message 4 of IKE

Phase1-Message-4

Phase1-Message-4

IKE Phase 1 (Main Mode): Preparation for Sending Messages 5 and 6

Before messages 5 and 6 can be sent, the two peers must calculate the DH shared secret. In addition, based on the exchanged nonce, the DH secret calculated, and the preshared keys stored on the two peers, three sets of keys must be generated that will be used to authenticate the two peers to each other as well as to encrypt subsequent IKE exchanges.Three keys are generated by each peer. These keys come out to be the same on both ends due to the nature of the DH exchange and the rest of the elements used to generate the keys. These keys are called session keys (SKEYs). The three keys being generated are as follows:

  • SKEYID_d— This key is used to calculate subsequent IPsec keying material.
  • SKEYID_a— This key is used to provide data integrity and authentication to subsequent IKE messages.
  • SKEYID_e— This key is used to encrypt subsequent IKE messages.Now that both ends have the keying material generated, the rest of the IKE exchange can occur confidentially using encryption done using SKEYID_e.

Note that for the keys to be generated correctly, each peer must find the corresponding preshared key for its peer. A number of preshared keys might be configured on both the peers for each of the many peers they communicate with. However, the ID payload identifying the peer’s IP address or the host name does not arrive until the next message is exchanged. Therefore, each peer must find the preshared key for its peer by using the source IP address from which it is receiving the ISAKMP packets. The reason for sending the ID payload later is to hide it by using encryption afforded by the keys that have been generated in this step. We will look at the problems this method can pose in the discussion of aggressive mode.

For the two peers to generate their SKEYIDs, they must first calculate their shared DH secret:

  • Calculation of the shared DH secret by the initiator

shared secret =(Xb)a mod p

  • Calculation of the shared DH secret by the responder

shared secret =(Xa)b  mod p

These two values end up being the same, gab, as discussed earlier.

After the DH secret has been calculated, the SKEYIDs can be generated. Figure 13-9 shows the session keys being generated by the initiator, and Figure 13-10 shows the session keys being generated by the responder.

Figure 13-9. Session Keys Being Generated by the Initiator

Session-Keys-Calculated

Session-Keys-Calculated

Figure 13-10. Session Keys Being Generated by the Responder.

Session-Key-Responder

Session-Key-Responder

IKE Phase 1 (Main Mode): Sending Message 5

Message 5 is sent with the usual ISAKMP header, but it contains two new payloads—the identity payload and the hash payload. Figure 13-11 shows message 5 of IKE being sent.

Figure 13-11. Sending Message 5 of IKE

Phase1-Message-5

Phase1-Message-5

Here are the various payloads included in the message:

  • Identity payload— This payload contains information about the identity of the initiator.This is in the form of the initiator’s IP address or host name.
  • Hash payload— The hash payload is calculated as shown in Figure 13-11. The hash is used for authentication purposes. The responder calculates the same hash on its end. If the two hashes come out to be the same, authentication is said to have taken place.

The two payloads are encrypted using SKEYID_E.

IKE Phase 1 (Main Mode): Sending Message 6

Message 6 is very similar to message 5 in that the responder sends the corresponding authentication material and its ID. Figure 13-12 shows IKE message 6 being sent.

Figure 13-12. Sending Message 6 of IKE

Phase1-Message-6

Phase1-Message-6

The contents of the packets are encrypted using SKEYID_E.

Completion of IKE Phase 1 (Main Mode) Using Preshared Keys

The main mode of IKE is completed using the material received in the last two messages of IKE exchange. The two sides authenticate each other by recalculating the hash and comparing it tothe one they received in the hash payload. If the two values are the same, the two sides areconsidered to have successfully authenticated.

The following list shows how the two peers use the authentication hashes received from eachother to verify each other’s authenticity:

  • Initiator authenticates the responder

                  Step 1. Decrypt the message using SKEYID_E.

                  Step 2. Find the configured PK-R ( pre-shared key or responder ) using ID_R.

                  Step 3. Calculate Hash_R on its own.

                  Step 4. If received Hash_R = self-generated Hash_R, authentication is successful!

  • Responder authenticates the initiator           

                  Step 1. Decrypt the message using SKEYID_E.

                  Step 2. Find the configured PK-I (pre-shared key or initiator) using ID_I.

                  Step 3. Calculate Hash_I on its own.

                  Step 4. If received Hash_I = self-generated Hash_I, authentication is successful!

At this point, the IKE SA is established, and main mode using preshared key authentication is complete. The next step, quick mode, is discussed in the next section.

IKE Phase 2 (Quick Mode): Preparation for Sending Messages 1 and 2

Just as the main mode was used to agree upon parameters for the IKE SA, quick mode is used to negotiate some of the parameters of IPsec security association. In our example, we will assume that the initiator has decided to use a property known as Perfect Forward Secrecy (PFS).

PFS

PFSis a property that the initiator of an IKE negotiation can “suggest” to the responder. Thissuggestion is made by sending a key exchange attribute payload during the first message of quickmode. If the responder agrees to do PFS, it continues the quick mode exchange. Otherwise, itreturns “Attributes Not Supported,” and the initiator can continue without PFS if it is soconfigured.

PFS is a property that forces the peers (if they agree to it) to generate a new DH secret duringthe quick mode exchange. This allows the encryption keys used to encrypt data to be generatedusing a new DH secret. This ensures added secrecy. If the original DH secret were to somehowget compromised, this new secret can ensure privacy for the data. Also, because the negotiationsfor the new DH are carried out using encrypted traffic, this new DH secret has an added elementof secrecy.

Note that it is not possible to negotiate the DH group the way it was offered and accepted in mainmode, because quick mode has only one exchange in which the DH exchange must take place.Therefore, the PFS DH group configured on both ends must match.

To ensure PFS, the two peers generate the numbers needed to perform DH exchange once again.This is done in exactly the same manner as was done in phase 1 of IKE. The following listdescribes how this is done by the two peers. The apostrophe on top of the various values is usedto signify the fact that these values are different from the DH values calculated in phase 1 of IKE:

Execution of DH by the initiator again to ensure PFS:

New nonce generated: Ni’

New DH public value = Xa’

Xa’ = gamod p

where g is the generator

p is a large prime number

a is a private secret known only to the initiator

  • Execution of DH by the responder again to ensure PFS:

New Nonce Generated: Nr’New DH public value = Xb’

Xb’ = gbmod p

where

g is the generator

p is a large prime number

b is a private secret known only to the responder

IKE Phase 2 (Quick Mode): Sending Message 1

Message 1 from the initiator to the responder contains payloads you have already seen in themain mode negotiation. The contents of the payloads are, of course, different. Note the presenceof the key exchange payload for requesting PFS. In addition, the new nonce is also sent across.

The hash payload contains a hash calculated on some of the elements that were agreed on inphase 1, as well as some of the payloads contained in this message itself. The purpose of thehash is to authenticate the peer again.

In addition, the message contains proposals and transform pairs for the IPsec SAs to be formed.The proposal payload includes the type of encapsulation to use, AH or ESP, and an SPI. The SPI isan interesting element. It is a 32-bit number that is chosen by the initiator to uniquely identify theoutgoing IPsec SA that is generated as a result of this negotiation in its database of securityassociations (it might be talking to a lot more than one peer at a time). The responder, uponseeing the SPI, makes sure that it is not the same as one of the SPIs it is using and starts using itfor its incoming IPsec SA. It also proposes an SPI for its outgoing (and the initiator’s incoming)SA, which the initiator agrees to after checking.

The associated transform payload includes parameters such as tunnel or transport mode, SHA orMD5 for integrity checking in ESP or AH, and lifetimes for the IPsec security association.

A new payload type introduced in this message is the ID payload. The ID payload contains theproxy identities on whose behalf the initiator does the negotiation. These are generally IP addresssubnets, but they can have more fields, such as port, too. In the case of a site-to-site IPsec setup with two gateways doing IPsec negotiations with each other, the proxy IDs are based on rulesdefined on the gateways that define what type of traffic is supposed to be encrypted by the peers.

Note that most of the message is still encrypted using one of the keys generated in phase 1.

For the sake of our example, assume that the initiator offers the following transform attributes forthe IPsec SA in one of the transform and proposal payload sets:

  • Encapsulation— ESP
  • Integrity checking— SHA-HMAC
  • DH group— 2
  • Mode— Tunnel

Figure 13-13 shows the first message of IKE quick mode being sent.

Figure 13-13. Sending the First Message of IKE Quick Mode

Phase2-Message-1

Phase2-Message-1

NOTE

Unlike IKE timeout negotiation, which is very strict, timeout negotiation in quick mode for the IPsec SA is more lenient. If the initiator proposes a transform that, all other things being acceptable, contains a timeout value that is higher than the timeout configured on the responder, the responder does not fail the negotiation. Instead, it simply returns the transform payload with the timeout modified to the lower value configured on it. The initiator then sets up the IPsec with this lower value.

Note that this behavior is different from the behavior shown in the negotiation of the IKE lifetime. A transform with a timeout higher than the one configured on the responder is unacceptable. If none of the transforms carry acceptable attributes,including an equal or lower timeout, the negotiation simply fails.

The difference in the behaviors is due to the fact that the IPsec SA timeout negotiation is done after authentication has taken place, whereas the IKE timeout negotiation is done between unauthenticated peers with no trust established.

Figure 13-14shows how the lifetimes are negotiated between two peers.

Figure 13-14. IKE and IPsec Lifetime Negotiation

Time-Negotiation

Time-Negotiation

IKE Phase 2 (Quick Mode): Sending Message 2

The second message is sent by the responder in response to the initiator’s first message. The fields are pretty much the same, with a few changes. Most notably, only the proposal and the transform payloads that were acceptable are returned to the initiator.

The proposal payload contains the SPI of the outgoing SA for the responder rather than the SPI that was sent by the initiator.

The hash payload contains the hash of only the payloads that are contained in this message. This hash acts as proof of the responder’s liveness because it contains a hash of the two latest nonces as well the message ID, proving to the initiator that the responder has indeed received its initialpacket and is ready to receive its encrypted messages.

The responder looks at the IDs offered by the initiator in the ID payloads and compares them tothe proxy IDs defined on itself. If its policy permits the IDs that are being offered by the initiator,the negotiation continues. Otherwise, a failure occurs.

Figure 13-15 shows message 2 of quick mode being sent.

Figure 13-15. Sending Message 2 of Quick Mode

Phase2-Message-2

Phase2-Message-2

IKE Phase 2 (Quick Mode): Preparation for Sending Message 3

Before the last message can be sent for quick mode, the two ends must use the information sentrelative to DH to generate a new DH secret and use this secret in conjunction with SKEYID_d andsome of the other parameters to generate the IPsec encryption/decryption keys. The following arethe steps taken by the two peers to generate the keys:

  • Initiator generates IPsec keying material

            Step 1. Generate new DH shared secret = (Xb’) mod p

    Step 2. IPsec session key for incoming IPsec SA = PRF (SKEYID_d, protocol(ISAKMP), new DH shared secret, SPIr, Ni’, Nr’)

        Step 3. IPsec session key for outgoing IPsec SA = PRF (SKEYID_d, protocol(ISAKMP), new DH shared secret, SPIi, Ni’, Nr’)

Responder generates IPsec keying material

    Step 1. Generate new DH shared secret = (Xa’)bmod p

    Step 2. IPsec session key for incoming IPsec SA = PRF (SKEYID_d, protocol(ISAKMP), new DH shared secret, SPIi, Ni’, Nr’)

      Step 3. IPsec session key for outgoing IPsec SA = PRF (SKEYID_d, protocol(ISAKMP), new DH shared secret, SPIr, Ni’, Nr’)

As soon as the IPsec key is generated, the tunnel is pretty much ready to become operational.Only one additional message is sent which concludes the quick mode negotiation.

Please note that two keys are generated on the initiator and the responder: one for the outgoing SA (which matches the key for the incoming SA key on the peer) and one for the incoming SA(which matches the key for the outgoing SA key on the peer).

IKE Phase 2 (Quick Mode): Sending Message 3

The last message of the IKE exchange is sent to verify the liveness of the responder. This is necessary for two reasons. The first reason is that the responder needs some way to know that the initiator received its first and only message of quick mode and correctly processed it. The second reason is to avoid a limited denial of service attack orchestrated by an attacker by replaying a first message of the quick mode exchange from the initiator to the receiver. By sending another message with the correct message ID and latest nonces hashed, the initiator proves to the responder that it has not only received its nonces but also is a live and current peer.Figure 13-16 shows how the final message is sent to the responder.

Figure 13-16. Sending Message 3 of Quick Mode

Phase2-Message-3

Phase2-Message-3

After message 3 has been received by the receiver, IPsec’s quick mode is concluded. All the goalsof IKE—authentication, negotiation of the encryption algorithm, and other attributes needed togenerate the encrypted packets—have been negotiated. Now the agreed-on IPsec SAs can beused to exchange encrypted traffic.

NOTE

Note that while it is not a requirement per the RFC, the responder can also send theinitiator a message similar to the message 3 discussed above to verify its liveliness.

Main Mode Using Digital Signature Authentication Followed by QuickMode Negotiation

Another important method of authentication used in IKE is digital signatures. We will discuss the use and workings of digital signatures in a later section. Here the changes in main mode of IKE negotiation are described. You might want to read the section on digital signatures (PKI) first and then come back to this section.

The only difference in the preshared and digital signatures method of authentication occurs in thefifth and sixth IKE messages that are sent. The first four messages of main mode and all thequick mode messages remain the same as in preshared key-based IKE negotiation.

Let’s look at the preparation and sending of messages 5 and 6 for digital signatures to appreciatethe difference.

IKE Phase 1 (Main Mode): Preparation for Sending Messages 5 and 6 Using Digital Signatures

The first difference that is quite evident is the way the SKEYs are generated when using digital signatures. Instead of generating PRF outputs with the preshared secret as one of the components, the PRF output is generated using the DH secret gab instead, along with the rest ofthe parameters, which are the same as the ones used in the preshared method. Obviously, at thispoint, anyone who knows how to do IPsec can negotiate (using a hit-and-miss method to discoverconfigured transforms and proposals) and have the keys generated, which would be valid on bothends. The effect that the preshared secret has, of limiting successful negotiations to those peersthat have the same preshared key configured, is not visible in this method up until this point. Inorder to achieve the same level of control when using digital certificates, a successful negotiationis restricted by either manually setting up each peer with the public key of the peers to which it isallowed to connect or enrolling each peer in a certificate authority (CA) server with a certainorganization name. All peers to which the peer is allowed to connect must enroll with the samecertificate authority server and belong to the same organization. Messages 5 and 6 of theexchange contain the certificates of the two peers. These certificates contain the identity of thetwo peers, including the organization ID.

Figure 13-17shows the mechanism through which the keys are generated on the initiator in thedigital signatures method of conducting main mode negotiation.

Figure 13-17. How Keys Are Generated on the Initiator in the DigitalSignatures Method of Main Mode Negotiation

Signatures-Key-Generation

Signatures-Key-Generation

Figure 13-18shows the mechanism through which the keys are generated on the responder in the digital signatures method of conducting main mode negotiation.

Figure 13-18. How Keys Are Generated on the Responder in the DigitalSignatures Method of Main Mode Negotiation

Signature-Key-Generation-Responder

Signature-Key-Generation-Responder

IKE Phase 1 (Main Mode): Sending Message 5 Using Digital Signatures

The message 5 sent by the initiator contains two new payloads in addition to the ones we have already looked at—signature and certificate.

Signature Payload

This payload contains the initiator’s signature. In general, a signature is a known value encrypted with an individual’s private key. A signature provides non repudiation, meaning that because data encrypted using a private key can only be decrypted using its corresponding public key, the sender of the data cannot back out of admitting that he or she sent the data. No one else but that person possesses the private key (corresponding to the public key) that was used to decrypt the data. Obviously, the assumption is that the data is known a priori to both the sender and the receiver. In the case of message 5, this information is indeed a combination of information that has either already been exchanged between the two parties or that has been generated based on exchanged information and is the same on both sides (for example, the SKEYID).

The hash that is encrypted using the private key is generated by hashing the accepted proposaland transform payloads along with certain other values, as shown in Figure 13-19.

Figure 13-19. Initiator Sending Message 5 in the Digital Signatures Method of Main Mode Negotiation

Authentication Material

Authentication Material

Certificate Payload

We will discuss the format of a certificate in detail in the section on IKE authentication methods.For now, it is sufficient to say that the certificate contained in the certificate payload is tied to aunique host name (or some other similar attribute) that is the sender’s host name. The certificatealso contains the sender’s public key, which is used to decrypt the signature sent in the samemessage. The certificate is issued for a certain organization ID. Both the IPsec peers must belongto the same organization, and both must trust the CA issuing the certificate for the certificate tobe acceptable to both of them.

Figure 13-19shows the initiator sending message 5 in the digital signatures method of conductingmain mode negotiation.

IKE Phase 1 (Main Mode): Sending Message 6 Using Digital Signatures

The message 6 sent by the responder to the initiator is pretty much the same as the messagesent by the initiator. It contains the certificate of the responder and the signature generated usingthe private key of the responder and the corresponding values stored on it through the IKEexchanges so far.

Figure 13-20 shows the responder sending message 6 in the digital signatures method ofconducting main mode negotiation.

Figure 13-20. Responder Sending Message 6 in the Digital Signatures Method of Main Mode Negotiation

IKE Phase 1 (Main Mode): Completion of Authentication Using Digital Signatures

As soon as both the initiator and responder have compared the ID of the peer with the ID set up in its configuration by the system administrator as the ID to which it is allowed to establish a peering relationship, the authentication process using digital signatures is completed by both the initiator and the responder. This is done by decrypting the encrypted Hash_I using the public keyof the other side received through its certificate. The hashes are then compared in a manner similar to the method used in preshared authentication to verify that they are the same. If theyare, the two sides are assumed to be authenticated.Initiator authenticates the responder

Step 1. Decrypt the message using SKEYID_E.

Step 2. Decrypt Hash_R using Pub_R.

Step 3. Calculate Hash_R on its own.

Step 4. If received Hash_R = self-generated Hash_R, authentication is successful!

Responder authenticates the initiator

Step 1. Decrypt the message using SKEYID_E.

Step 2. Decrypt Hash_I using Pub_I.

Step 3. Calculate Hash_I on its own.

Step 4. If received Hash_I = self-generated Hash_I, authentication is successful!

As soon as main mode has been completed as just described, quick mode goes through exactly the same steps as in the preshared authentication method.

Tags: , ,