IPSec (Internet Protocol Security)

This page describes the IPSec protocol.

Topics this page:

Site Map  
About this site  

The IPSec Protocol

The security template named "High Secure" configures IPSec to be used on a network.

IPSec [RFC 2404] can be used alone to secure intranet traffice or with ESP for authentication and L2TP (never PPP) for tunneling to create a VPN (albeit with higher overhead).

IPSec operates at the Transport OSI layer 3 (above the Network layer 2) — transparent to applications. Its 3 components:

1. AH (Authentication Header) [RFC 2402] are not encrypted for confidentiality, but both IP packet 51 headers and payloads are signed with a HMAC (Hash Message Authentication Code).

   AH maintains two SA's - inbound and outbound.

2. ESP (Encapsulated Security Payload) [RFC 2406] performs both Authentication and (optionally) DES-CBC or 3DES Encryption IP packet 50. So ESP packet data is not visible to Netmon.

   ESP maintains two SA's - inbound and outbound.

3. For key management, IPSec ensures data integrity using two Mechanisms: manual keying for a VPN with a small number of sites and IKE for automated key management of VPNs covering a large number of sites or supporting many remote users.

   IKE (Internet Key Exchange) [RFC 2409] builds Security Associations (SAs) through UDP port 500 with Internet key exchanges using SPI unique identification numbers. It uses two protocol phases:

   1. ISAKAMP (Internet Security Association and Key Management Protocol) [RFC 2408] defines the framework for a logical Security Association (SA) agreement on Protocols between two clients, one direction at a time. It uses IANA-defined identifiers ("Magic Numbers") to crate a shared secret key.

   2. Oakley Key Determination Protocol [RFC 2412] uses the DH key exchange algorithm, which combines public information with secret information to generate a shared secret over an unsecured namespace. Oakley also supports PFS (Perfect Forward Secrecy) so keys can't be regenerated. It has two modes:
      1. Main mode - New key generation material and new encryption key
      2. Quick mode - Already have key generation material and need new encryption key

      To enable Oakley logging, use regedit to add "Debug" REG_DWORD value 1 on HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\ PolicyAgent\Oakley

   Each SA vector defines the source and destination IP, and whether it's inbound or outbound. A SA bundle is formed when two protocols travel through a session pipe.

IETF's IPSec website provides drafts and RFCs related to IPSec cryptographic: 2085, 2104, 2403, 2404, 2405, 2407, 2410, 2451

White Paper: IP Security for Microsoft Windows 2000 Server provides an overview of IPSec, with scenarios.

Planning and Implementing IPSec in a Windows 2000 Network by Robert A. Eggleston, April 20, 2000

Book: "Internet and Intranet Security" by Rolf Oppliger

IKE attribute Numbers assigned by IANA

Using IPSec to Lock Down a Server at Microsoft's IPSec Home Page

IPSec Processing Steps

1. A consumer sends a message to a service provider.
2. The consumer's IPSec driver attempts to match the outgoing packet's address or the packet type against the IP Filter.
3. The IPSec driver notifies ISAKMP to initiate security negotiations with the service provider.
4. The service provider's ISKAMP receives the security negotiations request.
5. Both principals initiate a key exchange, establishing an ISAKMP SA and a shared secret key.
6. Both principals negotiate the security level for the information exchange, establishing both IPSec SAs and keys.
7. The consumer's IPSec driver transfers packets to the appropriate connection type for transmission to the service provider.
8. The provider receives the packets and transfers them to the IPSec driver.
9. The provider's IPSec uses the inbound SA and key to check the digital signature and begin decryption.
10. The provider's IPSec driver transfers decrypted packets to the OSI Transport layer for further processing.

Articles from Microsoft's Knowledge Base

• Q252735 - How to Configure IPSec Tunneling in Windows 2000
• Q253740 - Predefined IPSec Policies Errors in Windows 2000 Help
• Q232817 - IPSec Default Policies May Overwrite Policies on an Imported Computer
• Q234320 - IPSec Policy applied after being deleted from Group Policy
• Q234580 - "Soft Associations" Between IPSec-Enabled and Non-IPSec-Enabled Computers
• Q253498 - How to Install a Certificate for Use with IP Security
• Q254257 - IPSec Off-Load for Intel PRO/100 S Server and Intel PRO/100 Management Adapters
• Q256284 - Ipsecmon.exe May Display Incorrect Information
• Q227339 - Unable to Configure IP Security Using the Unattend.exe Utility
• Q234581 - Description of Internet Protocol Security Troubleshooting Tools
• Q255857 - IPSec Offload Statistics Are Not Available
• Q254949 - Client-to-Domain Controller and Domain Controller-to-Domain Controller IPSec Support
• Q226442 - "Unable to Load Device Driver" Error with Damaged Tcpip.sys File
• Q231585 - Overview of Secure IP Communication with IPSec in Windows 2000
• Q231587 - Using the IP Security Monitor Tool to View IPSec Communications

  Use the Windows 2000 Ipsecmon.exe to monitor IPSec traffic. Its functions:

  • Is IPSec enabled on host
  • Displays current SAs on host
  • Displays whether the SAs are hard or soft

Configuring IPSec

To avoid setting up IPSec on a non-sensitive machine (end-user terminal server box) is first install a network VPN with IPSec 3DES.

To configure IPSec on Windows 2000 Pro or Server, from the MMC, File, add the IPSec Policy Management snap-in because it isn't installed in Windows 2000.

Access can be limited by IP filter or IPSec policy native to the OS, the listening port can be changed in the registry: 187623

Windows 2000 negotiates based on IP Security Policies :

• Client (Respond Only) — Communicate normally (unsecured). Use the default response rule to negotiate with servers that request security. Only the requested protocol and port traffic with that server is secured.
• Secure Server (Require Security) — For all IP traffic, always require security using Kerberos trust. Do NOT allow unsecured communication with untrusted clients.
• Server (Request Security) — For all IP traffic, always request security using Kerberos trust. Allow unsecured communication with clients that do not respond to request.

To view currently active assigned policy, open the Group Policy MMC console and view TCP/IP settings because work on IPSec was originally done beginning in 1992 by IEEE for IPv6. It's been adapted as RFC 2401 for real work with IPv4. Microsoft jointly developed IPSec for Windows 2000 with CISCO. The name appears only if the computer is running local IPSec policy. But if the computer is running policy assigned through Group Policy, the name is unavailable and cannot be edited.

There are soft and hard SAs. To invoke a configured hard SA, first reset an SA: first stop (not merely Refresh) traffic to the server.

IPSec Logging goes to the Security Log. You can change the local Audit Policy to include what you want logged. Some packets (licensing info and print job acknowledgments) are not encrypted: 275727

To map local client drives into the session and copy files over the encrypted session, use the RDPClip and Drmapsrv utilities from the resource kit: 309825 However, they don't work with the Advanced client (the XP version, runable on W2K): 278139

IPSec (Internet Procotol Security) Policies

Part of Domain policies:

Policy	Description	Recommended	Default
Client (Respond Only)	Communicate unsecured. Only the requested protocol and port traffic with that server is secured. This is the default response rule to negotiate with servers that request security.
Secure (Request Secuirty)	For all IP traffic, always request security using Kerberos trust. Allow unsecured communication with clients that do not respond to request.
Secure Server (Require Secuirty)	For all IP traffic, always require security using Kerberos trust. Do NOT allow unsecured communication with untrusted clients.