Security

The OCS Edge Server: how many NICs do I need?

Devin L. Ganger — Thu, 14 Aug 2008 18:01:26 GMT

There are a lot of people out there who want to try to get around Microsoft's recommended configuration for the OCS Edge Server roles. For whatever reason, they don't like the thought of have two network interfaces, one on a publicly routable IP network, the other on the private network. I've talked in the past about some of the reasons why this configuration is not only recommended, but actually a good idea, but let's just say it took a lot of talking and thinking before I accepted that notion.

MVP Jeff Schertz has done a fantastic job of walking through the various permutations people have come up with, separating what will work from what won't, and explaining the pros and cons of each variant. I highly recommend this post.

I also want to amplify a point he makes: having multiple interfaces (whether physical or virtual) on the same subnet will cause interesting and otherwise inexplicable weirdness on a Windows machine. I'll write up the situation I'm seeing in a bit (not OCS!), but let me be clear: it's caused me all sorts of problems. Run, do not walk, away from any "solution" that requires this.

First Look at Microsoft Online Services: the Sign-In tool

Devin L. Ganger — Mon, 28 Jul 2008 18:30:18 GMT

Continuing from my previous post on MOS...

I didn't really mention this in the previous post, but MOS is designed to provide a hosted alternative to the server-side applications. One of the goals is to continue working with existing native clients and client access methods, so (for example) you can access your Exchange Online mailbox through OWA (running from MOS), through Outlook, or even through EAS/Windows Mobile. In order to do this, though, your client applications need to know how to talk to MOS and provide the proper credentials.

You can do this the hard way or the easy way. The hard way is running around and reconfiguring each application by hand and teaching your users how to use a separate set of credentials. The easy way is to use the MOS Sign-In tool, a little .NET 3.0 application that runs on the client desktop. It interacts with Outlook 2007 RTM/SP1, LiveMeeting 8, and IE7+.

When this application is run, it will invite the user to logon to MOS. The first time they do so, they're required to change their password. It then detects the apporpriate applications, offers to configure them to work with MOS, and then just sits quietly on the desktop, providing a seamless SSO experience.

To be continued...

These are not the solutions you're looking for

Devin L. Ganger — Fri, 27 Jun 2008 03:18:47 GMT

As IT professionals, we are more than often prone to fall to the perils of magical thinking. (I'm sure this is a side-effect of being human, which is a pesky and bothersome condition I will have to do something about one of these days.) Magical thinking in this context is when we have not internalized the intricacies of a problem and instead rely on formulas rather than true understanding to come up with solutions.

At one ISP I used to work at, we had a glorious reclaimed piece of technology, an Auspex NS-5500 file server. Every now and then on reboot, this old beast of a machine would fail to boot up; the cure was to open the cover over the drive cage and give it a good swift whack. We all assumed that this was because one of the drive connectors was a bit loose, but when our "magic" fix failed to work one night I discovered that it was in fact because one of the screws holding things in place was missing, allowing the drive bay to sag just a tiny bit. It was this tiny bit of sag that put just enough stress on the connector for drive 0. Had we actually opened the case up earlier, we'd have been able to solve the problem -- and prevent a year of whacking the server.

All too often, I see magical thinking in the field of security. Case in point: I recently heard about a gentleman who has a client that is requesting ETRN support be added back to Exchange 2007, either natively or through an add-on. They want to deploy the Edge role in their DMZ, have it queue up mail for the internal organization, and then have their Hub Transports (in the internal protected network) initiate a connection out to de-queue the messages using the ETRN SMTP extension. The reason they want this is that they've done due diligence and read some very thorough documents about computer network zones and have come to the conclusion that all network connections must be initiated from the most secure network. This, they say, removes the threat of malware taking over the Edge server in the DMZ and allowing an attacker to use it as a launching point to the protected network.

Now, the recommendation for connections to be initiated from a more secure network to a less secure network is a good general baseline to follow when it makes sense. However, it is not realistic in all cases (if we followed this to the letter, nobody would be able to receive e-mail from external senders except through random polling of Internet SMTP hosts, which is not at all scalable). This is doubly true if you don't understand how the underlying protocols work. Case in point: ETRN, defined by RFC 1985, "SMTP Service Extension for Remote Message Queue Starting". Quoting from section 3, "The Remote Queue Processing Declaration service extension" (emphasis added):

To save money, many small companies want to only maintain transient connections to their service providers. In addition, there are some situations where the client sites depend on their mail arriving quickly, so forcing the queues on the server belonging to their service provider may be more desirable than waiting for the retry timeout to occur.

Both of these situations could currently be fixed using the TURN command defined in [1], if it were not for a large security loophole in the TURN command. As it stands, the TURN command will reverse the direction of the SMTP connection and assume that the remote host is being honest about what its name is. The security loophole is that there is no documented stipulation for checking the authenticity of the remote host name, as given in the HELO or EHLO command. As such, most SMTP and ESMTP implementations do not implement the TURN command to avoid this security loophole.

This has been addressed in the design of the ETRN command. This extended turn command was written with the points in the first paragraph in mind, yet paying attention to the problems that currently exist with the TURN command. The security loophole is avoided by asking the server to start a new connection aimed at the specified client.

See the problem? ETRN was not designed to solve a security problem; it was designed to solve a financial problem back in days when always-on bandwidth was a lot more expensive and most ISPs metered traffic. It masquerades as solving a security problem only because it's designed to avoid a loophole in an insecure and exploitable feature. As a result, ETRN won't solve the problem these people want it to solve; all it does is tell the system in the DMZ to initiate a new connection to the Hub Transport servers. It doesn't reuse the existing connection initiated by the Hub Transport servers. They can't use a firewall rule to block outgoing access from the Edge to the Hub Transport and be safe, because they'll cut off all incoming traffic.

However, let us for a moment assume that it did work the way they wanted it to: my Hub Transport initiates an outbound SMTP session to the Edge. In this session, HT is the SMTP client, ET is the SMTP server. As soon as HT issues the ETRN command, they still have to swap roles -- HT is now using the SMTP server code paths, while the ET is using the SMTP client code paths. Any theoretical vulnerabilities that are in the HT SMTP implementation are still going to be there, still exposed to the message traffic about to be sent down the connection, still open to exploitation.

This is the magical thinking: firewalls and a DMZ will protect my traffic. This is not true; firewalls and networks zones are two components of a complete security plan. Neither firewalls nor network zones can protect legitimate traffic, nor are they designed to; they are designed to allow you to designate which traffic is legitimate. If you want to secure that traffic, you need to turn to other measures.

A certificate roundup

Devin L. Ganger — Fri, 09 May 2008 23:55:07 GMT

Certificates are one of the biggest issues I keep hearing about with Exchange and OCS, and apparently I'm not the only one. Fellow MVP Michael B. Smith has recently posted two blog articles on certs: how to use SAN certificates with ISA 2006 and other certificate limitations. However, he's got a couple of points on the second article that I'm confused about:

According to this announcement on the Windows Mobile team blog, Windows Mobile 6.0 and up do in fact support wildcard certificates.
The first point he makes is also head-scratcher, because I've also heard this was an issue, but I'd also recently heard of a workaround for it:
1. In Outlook, go to the properties for your Exchange account (Tools, Account Settings, select your Exchange account and click Change) and click More Settings.
2. On the Connection tab, click Exchange Proxy Settings.
3. Look for the field Only connect to proxy servers that have this principal name in their certificate and make sure it's checked (you may need to check the Connect using SSL only checkbox first).
4. The value in this field should normally be set to msstd:server.external.fqdn, the FQDN the server is known as from the outside and that is the subject name of the certificate. So if my certificate was issued for 3Sharp, it would be msstd:mail.3sharp.com. To use this with a wildcard certificate issued to *.3sharp.com, this value would need to be set to msstd:*.3sharp.com.
  
  Let's try a diagram to make the point:

I'm doing more checking, trying to figure out what the deal is here; in the meantime, if you've got operational experience with either of these issues, please let me know.

At any rate, there's some more interesting factoids on certificates I've picked up:

If you want to use a certificate with the Exchange 2007 UM role, you need to have a certificate on the machine whose subject name matches the server's AD/DNS FQDN. It seems that you can't enable a certificate for the UM service using the Enable-ExchangeCertificate cmdlet if this does not match. Note that you can do this for other services, such as those hosted by the CAS role; the cmdlet performs different name checks on the certificate based on the services (SMTP, POP3, IMAP, HTTP, and UM) that you are enabling.
I've said it before, but it needs to be repeated: if you're not using the default self-signed certificate, simply use the Enable-ExchangeCertificate cmdlet to move all services to one or more additional certificates. Do not delete the default certificate; although in most cases Exchange will simply recreate it when the appropriate service is restarted, you can cause subtle errors that will take a while to figure out.
Learn more about certificate usage in Exchange in Creating a Certificate or Certificate Request for TLS.
And learn more about the Enable-ExchangeCertificate cmdlet.

More later!

Security and the OCS 2007 A/V Edge role

Devin L. Ganger — Fri, 11 Apr 2008 18:33:42 GMT

When people start digging into the specifics of the A/V Edge role in OCS 2007, they usually have a strong and immediate knee-jerk reaction something along the lines of, "No way!" (Mine was, "Oh, heck no!") This reaction is usually caused by learning one or more of the following deployment requirements:

Public IP address. The A/V Edge server needs to have a publicly routable IP address. This address must be publicly routable; you can't fudge it by giving it an IP address in a private range (10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16) and do any sort of NAT to it. 1:1 NAT or static NAT mapping won't do the trick here. You can and should have a firewall between it and the Internet, but it can't be doing any address translation.
Dual-homed. The A/V Edge server cannot be separated from the internal OCS servers by NAT. Therefore, if you're using a private address range and NAT in your internal network, you have to give the A/V Edge server a second network interface and IP address on routable, non-NAT address range. (Note, however, it doesn't have to be the same address range as the internal network, simply on an address range that is directly routable without NAT.)
20,0002 external ports. The external (publicly routable) interface needs to have the following ports opened to the Internet: UDP 3478, TCP 443, UDP 50,000-59,999, and TCP 50,000-59,999. Security people immediately look at the need to have 10,000 dynamic TCP ports and 10,000 dynamic UDP ports and have their head asplode in sheer instinctive security reaction.

I've personally reacted to all three of these requirements; I've yet to talk to a security-conscious IT professional new to OCS who hasn't. So what on Earth is Microsoft doing putting these requirements in place? Have they completely lost it about security?

In a word, no.

There are good reasons why these requirements are in place. Rather than go over them myself, however, let me simply direct you to this excellent post on the OCS team blog. If you have any questions, post them there and tell 'em I sent you. Note that to post questions on their blog, you need to first join their Community Server site. This is painless and easy; simply click the Join link in the upper right-hand corner, pick a username and password, provide your email address, and you're ready to go.

New form of spam

Devin L. Ganger — Sat, 01 Mar 2008 03:29:00 GMT

I came across an interesting article yesterday on a new form of spam: using webmail providers' Out-of-Office features to do a new type of backscatter spam. This is an excellent example of how unsecured messaging does not mix well with automated message generation capabilities. Any good Web developer can tell you that it's a bad decision to blindly accept and process untrusted input, and yet SMTP bots (that's what OOF functionality is at its core) do precisely that, thanks to the lack of a standard for verifying the authenticity of the sending identity and the integrity of the end-to-end message route. This is nothing new; this is the same variety of vulnerability that backscatter spam has been exploiting for years: target the NDR/bounce generation mechanism to do the dirty work for the spammers and send the paylod to the victim.

This new form of attack just underscores my growing conviction that our current system of email is going to be gradually supplanted by a variety of mechanisms for communicating with people outside of our organizations. There's too big of a disconnect between “enterprise” features that business want from email and the inherent limitations of the current store-and-forward mechanism SMTP is built upon. And no, I'm not one of those people who thinks that pay-per-email schemes are the answer; what works well for physical, tangible products becomes quickly unworkable for virtual communications.

I don't think there's going to be One True Successor for SMTP, nor do I see SMTP going completely away any time soon (just as Usenet, despite all predictions, still manages to hang on for certain applications and communities). Dependable synchronous communications modes such as instant messaging, voice, and video will, I think, begin taking up a lot more of the message trafrfic currently carried by email. By avoiding store-and-forward asynchronous mechanisms, you reduce the opportunities that attackers and spammers have to forge and inject illegitimate communications into your users' workspaces. Allowing users to decide which communications mode is best for them helps alleviate the pressure on email systems.

Passwords in the 21st century

Devin L. Ganger — Mon, 25 Feb 2008 06:08:00 GMT

I am sick and tired of the shoddy programming practices most companies still have in place today with their websites.

I can understand the desire to not provide certain types of downloads to users unless they have an account that can be tracked, especially (yes, Parallels, I'm looking right at you!) when they distribute updates as a completely new installer instead of an updater or service pack. I can understand why they justify the need to use a completely separate account management system instead of one of the many standards that are available, such as Windows Live (formerly known as Passport). I cannot understand, then, why they spend the development (and, one would hope, testing) effort to write a sloppy, poor authentication system that makes assumptions about the habits of the users. If you're going to spend that internal time and effort poorly, just pay the fee to Microsoft for Windows Live already and at least give your users one fewer set of credentials to remember!

I use passphrases everywhere I can these days, even for "throwaway" accounts on websites. I know the arguments for weaker security on them and agree with them as a personal choice for the user; the website should not be free to make the same assumptions. I'm tired of getting error messages because I've entered "too many characters" (turned out that 12 was too many for that particular website) or dared to use symbols instead of just numbers and letters. How dare I try to keep myself in the habit of using cryptographically strong (and easy to remember) passphrases everywhere!

These may seem like little things, but if developers aren't even getting these usability issues right because they favor "decreased complexity" (what, properly handling symbols in a text string is too hard to figure out how to do properly?), what assurance do we, the consumer, have of them getting bigger security issues right?

Fighting PKI inertia

Devin L. Ganger — Wed, 06 Feb 2008 00:35:00 GMT

I've noticed something for a while now -- people are really reluctant to install a proper PKI system. If you're a Windows-based organization, I have three words for you:

Get over it.

The Windows Certificate Service (WCS) is powerful and fairly easy to manage -- and it's included in Windows Server Standard or Enterprise versions.

For years, people have been complaining about the lack of security in various products. Well, Microsoft and other vendors have listened, and standards like TLS and Mutual TLS are now getting put into most of the new products and versions rolling out the door. However, in order to USE these standards and get the security, you must have certificates. You can spend a lot of money buying and installing and managing these certificates from third-party vendors or you can install WCS.

The most common objection I hear is, “But I can just use commercial certificates!“ True, you can. But now you're paying for every certificate and adding to the complexity of your deployment tasks. Rolling out Exchange 2007 or OCS 2007 with all the proper certificates is a lot easier when your servers have an internal WCS infrastructure to talk to -- requests are fulfilled almost immediately. You don't have to spend money on all those internal server certificates -- just the external-facing certs for machines that are talking to external clients or mobile devices.

Either way you choose to go, there are a few facts of life:

You WILL need to take time to learn how certificates and requests actually work. Knowing why you want to keep secure exported copies of certificates with their private keys associated is a good thing.
You WILL have to allocate time managing your certificates and infrastructure. Commercial certificates expire -- we at 3Sharp had a certificate expiration sneak up on us and disable OWA and EAS until we got it sorted out.
You WILL have to worry about certificate backups and processes. See the point about exported certificates.

Okay, here's a question -- would there be any interest in having me do a series of blog posts on the basics of certificate handling? I know there's good material out there, so I'd focus my stuff on common tasks and gotchas I've run across when deploying certificates for Exchange and OCS. If that sounds like something you'd want to see, drop me a comment.

A couple of quick links

Devin L. Ganger — Wed, 14 Nov 2007 21:27:00 GMT

I'm on my way out the door to fly to Chicago to present the last session of a Unified Communications show, but I want to give you a couple of quick tidbits:

I'd like to welcome former Exchange MVP and former Microsoftie Kevin Miller to the ranks of the 3Sharp bloggers. Kevin joined us over the summer and has just now started blogging.
I'm proud to share that I was selected to write the "MVP Article of the Month" for the November edition of the Microsoft TechNet Security Newsletter: Top 5 Exchange Server 2007 Security Best Practices.

Enjoy, and I'll chat with you all next week!

A quick Exchange 2007 advice retraction

Devin L. Ganger — Thu, 12 Apr 2007 21:22:00 GMT

I recently discovered that I've been unintentionally dispensing incorrect advice about a new Exchange 2007 Edge Transport feature. My shame at getting it wrong is only moderated by knowing I'm in good company. Fellow Exchange MVP Jim McBee recently posted to his blog on this same topic, and so I'll just quote him here:

The feature is just "safe senders"; safe sender list aggregation does not include the user's blocked senders. As Microsoft's Ross Smith pointed out to me, that is why it is called "safe sender list aggregation." :-)

So if you see me talking about how Edge does safe/blocked sender aggregation, ignore the "blocked sender" part, because that's my inner idiot poking out and saying hello.