Cryptography: Hashing & Encryption

Here are my brief notes on techniques to keep data integrity and confidentiality.

Cryptology consists of two professions and disciplines:

• Cryptographers practicing Cryptography — concealing information using mathematical transformations (hashing and encryption).
• Cryptanalysts practicing Cryptanalysis — revealing information hidden by cryptography using analytical and mathematical techniques, without the consent of the cryptographer.

Cryptographers' Homepages

Topics this page:

Site Map  
About this site  

Related Topics:

Secure Channels

Secure exchanges of data between a Consumer and a service Provider (what Microsoft calls principals) through a Session Pipe. Each is a communicant to the other. How secure a communication is depends on: keeping private keys secret, length of keys, and how often they are changed. (algorithm strength - reverse).

A secure channel is a means of data communication known (assumed) to be safe from intrusion and modification. Physical transport by a trusted messenger can be considered a safe channel. Authentication can be by voice recognition.

Encryption provides assurance of message confidentiality (that others cannot read it), This is provided by use of the HTTPS protocol using SSL certificates on the server.

To ensure message integrity (that the message was not changed enroute from the sender), the sender creates a digital signature. Digital signatures are like handwritten signatures, which are based on the idiosyncratic way one signs one's name.

But Digital signatures are much more difficult to forge than a hand-written signature since they are created by encrypting a hash of the message being sent using his/her private key. This makes it so the signed message cannot be altered without also invalidating the signature.

That private key is created using the sender's password plus unique identifying information about the sender (such as an email address, social security number, drivers license number, etc.). These keys are also called "Digital IDs".

Digitally signing a message provides a verifiable record of the transaction to the recipient that the sender cannot repudiate.

Websites use the Entrust TruePass FIPS 140-1 Level 1 certified Web portal security solution does not require installation of an internet browser plug-in because it provides an html page that transparently downloads a Java applet that creates digital signatures using a "profile" (epf) file pre-generated for a specific user and stored on the machine's local drive (preferrably a removeable diskette or USB drive).

Entrust TruePass 7.0 delivers support for a 3-key pair model that defines a separate key pair to perform encryption, authentication, and digital signatures, as required by IETF RFC 3039 and European Union Digital Signature Directive.

Explanation of this by the US Patent and Trademark Office and this company.

Encryption (Encipherment) Mechanisms

A Digital Envelope uses two or more layers of encryption:
1. Symmetric keys are based on a shared secret. The same key is used to encrypt and decrypt.

   The need to distribute the secure key makes symmetric key vulnerable to compromise through cryptoanalytic code breaking. So the key needs to be communicated via a courier using different medium of communication than the one used for exchange of encrypted data. This poses a logistical nightmare.

   Nevertheless, symmetric keys are used to establish sessions because they are quick to encrypt and decrypt. But the keys are changed during a session to limit time available for cryptoanalytic code breaking.

   One type is a codebook of codewords used instead of the word to be hidden. But they can be broken by analysis of known letter and word frequency repetitions in the language, such as the letter "e" and the word "the".

2. Asymmetric keys consists of a public and a private (secret) complement. The two are NOT just half of the same key (e.g., the first and second 64 bits of 128 bits). The word “Asymmetric” means “not equal” because it is a “Strong” encryption technique. The secret (private) key cannot be derived from the ciphertext or public key, even if the algorithm is known.

   If the private key is used to encrypt message payloads, only the public key can be used to decrypt the encrypted cyphertext.
   If the public key is used to encrypt, only the private key can be used to decrypt.

   I think the mathematical technique to make public key cryptography possible is one of the greatest inventions of the 20th century (by Whitfield Diffie and Martin Hellman in 1975) because the secret need never be shared. But it allows people who have no preexisting security arrangement to exchange messages securely.

Public and private Keys are stored in encrypted form on separate keyring files.

Cryptlib is a large open-source library of crypto functions.

RSA's Crypto FAQ

SafeTP is a seamless public-key encryption shareware tool that sits between FTP and your network stack, intercepting ftp commands to encrypt them before sending. The control channel alone or both data and control can be encrypted, if desired. Use this tool with any ftp sync tool. I had to configure my NAT firewall to allow a range of ports for PASV mode.

Network Security: Private Communication in a Public World by Charlie Kaufman, Radia Perlman, and Mike Spencer. (Prentice Hall, 1995) It contains a clear description of how DES works.

Firewalls and Internet Security: Repelling the Wily Hacker by William R. CHeswick and Steven M. Bellovin. (Addison-Wesley) written by senior researchers who maintained and redesigned AT&T's Internet connections.

MPPE (Microsoft Point-to-Point Encryption)

PPTP doesn't encrypt, so Windows 2000 uses stronger encryption method MPPE (Microsoft point-to-point encryption). Note this is data link, not end-to-end encryption that IPSec provides between enpoints of the tunnel.

MPPE requires EAP-TLS, MS-CHAPv1 or MS-CHAPv2 at one of three levels of encryption set in RRAS Remote access policy:

• 40-bit for backward compatibility,
• 56-bit for international use,
• 128-bit for North America.

Hashing Algorithms for Message Digests

HMAC (Hash Message Authentication Code) block is a hash value, much like a checksum. The "hash" is fixed-length message digest (commonly represented by m') is the result of the hashing function (represented by h) deriving a variable-length message (commonly represented by m).

A "good" hash function is one that very rarely maps two different data items to the same index, This is why jash functions are used to spread out the location of unique data (such as names) into a hash table for index into a daatabase table. Hashing eliminates the need to resort indexes.

Cryptographically secure digest functions make it so a user's password doesn't have to be stored "in the wild" but a digest of the password, which can be used to determine whether a user has logged on.

Hashing is not considered encrypted cipher:

1. A digest is not intended to be reversible, but a cipher is.

2. A digest function compresses the data, whereas a cipher generally does not. In fact, a cipher usually expands the data.

3. Although digest functions need no keys — as they scramble data by the nature of the mathematical transformation itself.

The digest of plaintext can serve as a Modification Detection Code (MDC).

The different hashing functions for different cryptographic standards

• MD5 (Message Digest 5) [RSA/PKCS RFC 1321] (an improvement to MD2) converts 512-bit (64 byte) blocks of cleartext to 128-bit (16 byte) tags. It's used within ESP and AH [RFC 2403]

• SHA-1 (Secure Hash Algorithm) outputs 160-bit (20 byte) message digests. It was designed for use with DSS (the Digital Signature Standard) by NIST and NSA [published in FIPS PUB 180-1 in 1995 to revise FIPS PUB 180 in 1993], who have kept secret its design criteria and the attack on the first version.

  It's more collision resistent than MD5 -- different messages are unlikely to converge on the same hash value. So it's used by IPSec.

• RIPEMD-128 and RIPEMD-160 is a 128-bit and 160-bit cryptographic hash functions tuned for 32-bit processors. It was developed as part of EU project RIPE (RACE Integrity Primitives Evaluation, 1988-1992), who have made this algorithm public and unpatented. RIPEMD-256 and RIPEMD-320 are optional extensions for applications requiring a longer hash result, as within ESP and AH [RFC2857]
• El Gamal, published by T. ElGamal in July 1985, produces ciphertext that is about twice as long as the plaintext.
• The Modular Arithmetic Secure Hash, Algorithm 2 (MASH-2)

The standard used by banks are defined by ANSI X9, which has a relationship with IEEE P1363 standard.

Introduction to Cryptography With Java Applets (Jones & Bartlett Publishers, Inc., 2003) by David Bishop (a professor Grinnell College) offers online Java applets to demonstrate important concepts to help undergrads learn to program cryptography algorithms from the ground up. Topics include large integer computing, linear cyphers, matrix cyphers, primality testing, and establishing keys and message exchange.

Cryptography for Internet & Database Applications (Indianapolis, Ind. John Wiley & Sons, 2002) by Nick Galbreath

A course in Number Theory and Cryptography by Neal Koblitz. (Springer-Verlag) is a graduate-level textbook on number theory and cryptgraphy.

A Cryptographic Companion by John J. G. Savard

J. Orlin Grabbe's testMD.java sample program calculates several hash functions.

Microsoft's Encrypting File System

Windows 2000 does not encrypt compressed, read-only, nor system files even though encryption can be selected for such files:

From Windows Explorer, right click a file or folder for its Properties. The General tab will have an Advanced... button if the drive is NTFS. Click it. Select Encrypt contents to secure data.

Notice Compress contents to save disk space is automatically unselected when Encrypt is selected because Windows 2000 does not encrypt compressed files.



To convert files into ciphertext, Microsoft Windows 2000 uses DESX, a modified version of DES.

• 40-bits in International editions using Microsoft Basic Cryptographic Provider v1.0
• 56-bits in the North American edition using Microsoft Enhanaced Crypographic Provider v1.0

To make Windows 2000 encrypt with three 128 bit keys, enable Triple DES (also called TDES or 3DES or DES3), install the hardened "High Encryption Pack add-on for Windows 2000" .
This can be used by all countries except those embargoed and debarred or otherwise controlled by various agencies of the US government.

Use the CIPHER.EXE utility to /Encrypt, /Decrypt /All files or /Specified directories in NTFS partitions. For a full list of options, including /I to Ignore errors, enter this at a command prompt:

cipher /?

During file encryption, Windows 2000 generates a random number called the FEK (File Encryption Key) using the file owner's public key. If the user does not have a key pair, one is generated on the fly and stored in the user's profile. The encrypted FEK is stored in the DDF (Data Decryption Field) for the file. Another copy of the FEK is encrypted with the public key of one or more recovery agents and stored in the DRF (Data Recovery Field) for the file. If the user's machine is not a member of a domain, the Local Administrator account is the designated recovery agent for all files encrypted on that machine. (this account's key pair is also automatically generated). In a domain environment, the default recovery agent is the Domain Administrator account. (This can, and typically should, be customized via group policy)

So before you encrypt, modify the Recovery Agent's default setup in which the Administrator account is the agent for locally encrypted data. Reassign the Recovery Agent role to a separate domain account so that in the event of failure, the designated domain account-based Recovery Agent can recover the data.

The LSAS (Local Security Authority Server) decrypts files by first decrypting the FEK in the DDF using the owner's private key or decrypting the recovery agent's FEK from the DRF. The decrypted FEK is then used to decrypt the encrypted file. This should be transparent process.

Encrypted files are not indexed by the Index Service.

Since data is still sent over a network in clear text, allowing it to be captured as it travels over the network, also use IPSec to secure network communications.

If an encrypted file is copied to a diskette or other non-NTFS (FAT32) volume, the file is automatically unencrypted into clear text! If a file is moved from an EFS file, it is left decrypted.

efsinfo.exe from the Resource Kit Security.cab file lists for encrypted folders and files their /Certificate thumbnail and /Recovery agent:

Documents and Settings: Not Encrypted
Lotus: Not Encrypted
MSSQL7: Not Encrypted
Program Files: Not Encrypted
RECYCLER: Not Encrypted
System Volume Information: Not Encrypted
WINNT: Not Encrypted

The Winadvapi is used to disable EFS for specific folders.

To disable EFS at the OU level, delete the Encrypted Data Recovery Agents policy. An empty policy would nullify the inidividual recovery policies for computers within the OU.

Microsoft's white paper: Encrypting File System

Q223316: Best Practice for Encrypting File System.

Step-by-Step Guide to EFS

Q230490: The Encrypted Data Recovery Policy for Encrypting File System

Encrypting File System for Windows 2000

EFS for Windows XP

Microsoft Corporation. Windows 2000 Server Resource Kit . Volume: Microsoft Windows 2000 Server Deployment Planning Guide. Redmond, Washington: Microsoft Press, 2000. Chapter 11, "Planning Distributed Security." includes a short discussion on EFS.

Microsoft Corporation. Windows 2000 Server Resource Kit . Volume: Microsoft Windows 2000 Server Distributed Systems Guide . Redmond, Washington: Microsoft Press, 2000. Chapter 15, "Encrypting File System."

Microsoft Corporation. MCSE Training Kit: Microsoft Windows 2000 Server . Redmond, Washington: Microsoft Press, 2000. Lesson 2 in Chapter 11, "Microsoft Windows 2000 Security."

Alternatives for MS-EFS

DriveCrypt uses 256 bit steganographic techniques (hiding data in music files).

BestCrypt includes source code.

Pointsec

Caveo is an entire anti-theft service

Utimaco's SafeGuard Easy

PGPDisk, now integrated with PGPfreeware 6.0.2i

Reflex Magnetics Datavault

GuardianEdge's Encryption Plus for Hard Disks (EPHD) uses for encryption Bruce Schneier's 160-bit Blowfish Advanced Encryption Standard (AES) in CBC mode with 256 bit keys. The disk key used to encrypt the data on the disk is randomly generated and stored encrypted under the disk key encryption key (Disk KEK) derived from the username and password.

Users enter their password in the product, which then automatically supplies the Windows password.

The main reason why companies buy EPHD is that it has a remote corporate key recovery mechanism.

The product supports a one-time password in case users get locked out of their data. It can run on NT and Windows 9x.

Lost Password Recovery

On Windows 2000, the Encrypted Data Recovery Agents policy lists users authorized to recover data from encrypted files when the owner's private key is lost. The Administrator is in there by default upon activation.

The RID Master manages the distribution of sequential relative identifiers.

The first EFS log created is automatically named efs0.log.

Security Questions

• What was the name of your high school?
• What is your pet's name?
• What is your all-time favorite sports team?
• What is your father's middle name?
• What is your mother's maiden name?
• What make was your first car?
• Who was your first crush in school?
• What is the name of your favorite teacher?
• More at About People

Microsoft's Threats and Countermeasures Guide v2.0 published 12/27/2005 lists Security Settings in Windows Server 2003 and Windows XP.
Microsoft's Windows Server 2003 Security Guide
Microsoft's Windows XP Security Guide

Passware Kit from LostPassword.com performs password recovery.

Ciphers

Data Encryption Standards

Developoment on DES (Data Encryption Standard) started in 1975 and was standardized in 1981 as ANSI X3.92. FIPS PUB 46-1, which uses fixed-length 56-bit key and 8 bit parity check in its CBC (Cipher Block Chaining) algorithm [FIPS PUB 81] in 64 bit blocks.

Triple DES (3DES) recursively encrypts using DES.

Because of Massive Multiprocessing (MMP) brute-force cryptanalysis (the interception and attempted recovery of cleartext from a secured exchange), and use of single keys, DES is no long considered secure. That's why the US allows its export unhindered.

Other Symmetric Ciphers

The RC in RC2, RC4 ('87), RC5, RC6 ('98) - variable key-length ciphers -- is for Rivest Cipher because they were developed by Ron Rivest of RSA Data Security.

IDEA is a 128-bit International Data Encryption Algorithm used in open source Pretty Good Privacy (PGP)

Skipjack is an 80-bit key and 32 rounds on 64-bit blocks. It's used by the NSA for sensitive but nonclassified data.

Blowfish, developed by USC professor Bruce Schneier and author of many popular books, including Applied Cryptography (), uses a variable-round block cipher using any length keys up to 448 bit. It's also a 2001 movie starring John Travolta and Halle Berry.

Twofish is faster because it supports 128 bit blocks and 128, 192, or 256 bit keys. It's used with smartcards.

Rijnadael has similar capabilities. On October 2, 2000, NIST chose it for their new Advanced Encryption Standard (AES).

The DSA (Digital Signature Alogorithm), invented by David Kravitz and introduced by NIST, is the standard public-key encryption method in Linux.

• Newsgroup: sci.crypt

• ZDNET Techguide on Encryption (Registration and pesky survey response required)

Smart Cards Cryptoki

PKCS #11 compliant devices such as smart cards. create secure Java applets that offer token-based authentication and digital signatures provided by smartcards. RSA's PKCS #11 - called Cryptoki (pronounced crypto-key) - short for cryptographic token interface - is a low-level CAPI Cryptographic API (Application Programming Interface). currently at version v2.11 Cryptoki follows a simple object-based approach, addressing the goals of technology independence (any kind of device) and resource sharing (multiple applications accessing multiple devices), presenting to applications a common, logical view of the device called a cryptographic token.
• Mechanism: a cryptographic algorithm
• token: module which implements the mechanism
• slot: abstract adaptor which holds a token
• session: logical connection between an application and token
• objects: specific data types (e.g. DES3 secret key)

Security hardware offers significant security advantages over pure software applications. For example, portable tokens such as smartcards reduce the headaches of managing passwords, by enabling users to carry digital certificates on a credit card-size device.

Cryptographic accelerators can greatly improve the performance of security-bound systems, by offloading expensive crypto operations to a dedicated processor - while ensuring that vital information, such as private keys, never leave the confines of the hardware device.

Security Officer (SO) must initialize token. C_InitToken initializes token by destroying objects and denying normal user access until PIN is set

Token users must login with PIN before accessing private objects.

Applications cannot directly manipulate Cryptoki objects and contexts, which are created, manipulated, and destroyed beneath the API.

CDSA

Unlike the Common Data Security Architecture (CDSA), Cryptoki is not an entire architecture consisting of auxiliary services such as data storage, libraries, and memory management for applications. CDSA Libraries include: Trust Policy, Data Library, Certificate Library, Elective Library (Open Group CDSA Specification c707, ISBN 1-85912-194-2)

Eracom CSA8000 hardware and API Cprov for C and Jprov for working with Java Cryptography Extension (JCE), a standard extension to the Java Cryptography Architecture (JCA).

Applications requiring Crypto Keys

Applications that require secure hardware, including access control via a user token, and cryptographically intensive applications, such as SSL server applications. Other examples include:
• Online banking and financial transactions
• Extranet and B2B marketplace applications
• Document encryption and Signing
• Secure messaging with S/MIME
• Web-based access control via SSL
• PKI infrastructure components
• PKI server and middleware applications
• Web server and Java application server acceleration

More: Resources

• nCipher cryptographic HSM cards are used by Microsoft.
• The Microsoft Internet Security Framework: Technology for Secure Communication, Access Control, and Commerce Microsoft's Introduction The Microsoft® Internet Security Framework (MISF)
• $50 Cypherus provides drag-and-drop encryption using 448-bit blowfish encryption, 4096-bit Diffie-Hellman key exchange, and File Shredding levels that exceed DoD standards.

  The letters of the English alphabet, listed in order of frequence of use:

  ETAONRISHDLFCMUGPYWBVKXJQZ

• Electronic Privacy Information Center advocates privacy.

Cryptographic engines for use within IIS:

• $149 ABCCrypto
• $29 ASPEasyCrypt
• $249 ASPEncrypt
• $199 ASP MagicCrypt
• cASPer Encryption Package
• Cast 128
• DigiSign 2.0
• DynuEncrypt
• EncryptCC
• Hash
• Polar Crypto