One of the cool things you can do with OCS is connect your internal organization to various public IM clouds (MSN/Windows Live, Yahoo!, and AOL) using the Public Internet Connectivity, or PIC, feature. As you might imagine, though, PIC involves lots of fiddly bits that all have to work just right in order for there to be a seamless user experience. Recently, lots of people deploying OCS 2007 R2 have been reporting problems with PIC – specifically, in getting connectivity to the AOL IM cloud working properly.
Background
It turns out that the problem has to do with with changes that were made to the default SSL algorithm negotiations made in Windows Server 2008. If you deployed OCS 2007 R2 Edge roles on Windows Server 2003, you’d be fine; if you used Windows 2008, you’d see problems.
When an HTTP client and server connect (and most IM protocols use HTTPS or HTTP + TLS as a firewall-friendly transport[1]), one of the first things they do is negotiate the specific suite of cryptographic algorithms that will be used for that session. The cipher suite includes three components:
- Key exchange method – this is the algorithm that defines the way that the two endpoints will agree upon a shared symmetric key for the session. This session key will later be used to encrypt the contents of the session, so it’s important for it to be secure. This key should never be passed in cleartext – and since the session isn’t encrypted yet, there has to be some mechanism to do it. Some of the potential methods allow digital signatures, providing an extra level of confidence against a man-in-the-middle attack. There are two main choices: RSA public-private certificates and Diffie-Hellman keyless exchanges (useful when there’s no prior communication or shared set of trusted certificates between the endpoints).
- Session cipher – this is the cipher that will be used to encrypt all of the session data. A symmetric cipher is faster to process for both ends and reduces CPU overhead, but is more vulnerable in principal to discovery and attack (as both sides have to have the same key and therefore have to exchange it over the wire). The next choice is streaming cipher or cipher block chaining (CBC) cipher? For streaming, you have RC4 (40 and 128-bit variants). For CBC, you can choose RC2 (40-bit), DES (40-bit or 56-bit), 3DES (168-bit), Idea (128-bit), or Fortezza (96-bit). You can also choose none, but that’s not terribly secure.
- Message digest algorithm – the message digest is a hash cipher used to create the Hashed Message Authentication Code (HMAC), which is used to help verify the integrity of the cipher. It’s also used to guard against an attacker trying to replay this stream in the future and fool the server into giving up information it shouldn’t. In SSL 3.0, this is just a MAC. There are three choices: null (none), MD5 (128-bit), and SHA-1 (160-bit).
Problem
Windows Server 2003 uses the following suites for TLS 1.0/SSL 3.0 connections by default:
- TLS_RSA_WITH_RC4_128_MD5 (RSA certificate key exchange, RC4 streaming session cipher with 128-bit key, and 128-bit MD5 HMAC; a safe, legacy choice of protocols, although definitely aging in today’s environment)
- TLS_RSA_WITH_RC4_128_SHA (RSA certificate key exchange, RC4 streaming session cipher with 128-bit key, and 160-bit SHA-1 HMAC; a bit stronger than the above, thanks to SHA-1 being not quite as brittle as MD5 yet)
- TLS_RSA_WITH_3DES_EDE_CBC_SHA (you can work out the rest)
- TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
- TLS_RSA_WITH_DES_CBC_SHA
- TLS_DHE_DSS_WITH_DES_CBC_SHA
- TLS_RSA_EXPORT1024_WITH_RC4_56_SHA
- TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA
- TLS_DHE_DSS_EXPORT1024_WITH_DES_CBC_SHA
- TLS_RSA_EXPORT_WITH_RC4_40_MD5
- TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5
- TLS_RSA_WITH_NULL_MD5
- TLS_RSA_WITH_NULL_SHA
Let’s contrast that with Windows Server 2008, which cleans out some cruft but adds support for quite a few new algorithms (new suites bolded):
- TLS_RSA_WITH_AES_128_CBC_SHA (Using AES 128-bit as a CBC session cipher)
- TLS_RSA_WITH_AES_256_CBC_SHA (Using AES 256-bit as a CBC session cipher)
- TLS_RSA_WITH_RC4_128_SHA
- TLS_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256 (AES 128-bit, SHA 256-bit)
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384(AES 128-bit, SHA 384-bit)
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P521(AES 128-bit, SHA 521-bit)
- TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256(AES 256-bit, SHA 256-bit)
- TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384(AES 256-bit, SHA 384-bit)
- TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P521(AES 256-bit, SHA 521-bit)
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256 (you can work out the rest)
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P521
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P521
- TLS_DHE_DSS_WITH_AES_128_CBC_SHA
- TLS_DHE_DSS_WITH_AES_256_CBC_SHA
- TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
- TLS_RSA_WITH_RC4_128_MD5
- SSL_CK_RC4_128_WITH_MD5 (not sure)
- SSL_CK_DES_192_EDE3_CBC_WITH_MD5 (not sure)
- TLS_RSA_WITH_NULL_MD5
- TLS_RSA_WITH_NULL_SHA
Okay, so take a look at line 20 in the second list – see how TLS_RSA_WITH_RC4_128_MD5 got moved from first to darned near worst? Yeah, well, that’s because AES and SHA-1 are the strongest protocols of their type likely to be commonly supported, so Windows 2008 moves those to the default offered. Unfortunately, this causes problems with PIC to AOL.
Solution
Now that we know what the problem is, what can we do about it? For the fix, check out Scott Oseychik’s post here.
[1] HTTPS is really Hop Through Tightened Perimeters Simply – aka the Universal Firewall Traversal Protocol.