A Little Something on NTLM Authentication

At work I've been tasked with creating a channel of communication between an Oracle database and an IIS web service. Turns out Microsoft doesn't like to play fair with, well, anyone but Microsoft. I wrote up this blob to describe the awesome fun it will be to replicate this process.

NTLM Authentication
NTLM is a proprietary network authentication protocol that is implemented in various Windows network services. The standard NTLM authentication consists of three messages that will be used in the authentication handshake. Although there are many variations of the NTLM authentication, for this explanation we will be using NTLMv1 to authenticate packets containing SOAP content sent to SQL Server Reporting Services, a server using IIS.

The overall procedure is as follows:
1) The client sends a HTTP request with SOAP content attached.

2) Since the request does not provide the required authentication, the server responds with supported authentication methods in the response header.

3) The client receives this packet that has a status of 401 (unauthorized) and checks the headers for supported authentication methods. If the client finds a header named “WWW-Authenticate” with the value “NTLM” it proceeds to build the first part of the NTLM handshake. A Type1 message is sent from the client to the server as a header in another HTTP request. The Type1 message contains information on the domain and workstation along with flags denoting the version of NTLM supported and content type.

4) The server receives this packet, decodes the type1 message and responds with a packet with 401 status and a Type2 message. A Type2 message contains information about the network environment such as server and domain NETBIOS and DNS. More importantly, the Type2 message contains an 8-byte challenge which will be used by the client to prove that it knows the password without actually sending it across the network.

5) The client takes the challenge from the Type2 message and builds the Type3 message, the final part of the handshake. This is the first time that the username and password of the service are used. The username is not obfuscated, however the password is. The password is encrypted to create what is called the LM response.

Creating the LM response:
a. The password in uppercase is null padded to 14 bytes then split in two halves.
b. For each 7-byte half, after every 7th bit a parity bit is added, expanding the overall length to 8 bytes.
c. Each half is then used as a key to DES-encrypt the ASCII string “KGS!@#$%”.
d. The two cipher text values are then concatenated to create a 16-byte LM hash.
e. This hash is then null padded to 21 bytes and split into three 7-byte parts.
f. For each 7-byte part, parity is added expanding the overall length of each part to 8 bytes.
g. Using each of the three parts as individual keys, the challenge from the Type2 message is DES-encrypted.
h. Concatenate the three resulting cipher text pieces into one to create your LM response.

6) After the server receives the Type3 message and checks the username and password against its database, it returns a packet with status 200 (OK) and the requested content attached.

NTLM Token Structure:
The NTLM messages that at passed in HTTP packets are base 64. After casting the varchar token to raw and decoding it from base 64, you have a hex result that consists of both a fixed and variable length part. The fixed length part contains NTLM standard variables and references to where the viable length variables can be found. Strings are ASCII and numbers are little-endian (least significant first). The hex starts off with the signature “NTLMSSP” and then the type being 1-3. The rest of the content varies between types and is out of the scope of this document but is best described here: http://davenport.sourceforge.net/ntlm.html.