E-mail

First look at Microsoft Online Services: adding domains

Devin L. Ganger — Mon, 28 Jul 2008 18:21:46 GMT

I'm at an airlift here in Redmond for the new Microsoft Online Services (MOS), Microsoft's hosted services platform. Right now, MOS offers a combination of hosted Exchange (OWA, Outlook, and even EAS!), hosted SharePoint, and Live Meeting. We've just gone through an overview of the service, and it looks cool -- enough so that I'm now seriously considering switching my personal domains over to it (especially since they offer the ability to synchronize with your Active Directory deployment).

MOS is currently in beta and you can go sign up for a time-limited trial. There's only a certain number of trial accounts active at any given time, so your trial request may not be provisioned immediately; however, you can go to https://mocp.microsoftonline.com and sign up for one. You'll need a Windows Live account.

As you might imagine, MOS allows you to associate one or more DNS domains with your online account. When you register for your account, you're asked for a domain. This domain is not verified and, in fact, seems to be used simply as an internal administrative tag -- once your account and service is set up, you have to specifically add DNS domains. Adding them is a fairly simple process:

Register your domain name with a registrar.
Provision your domain with a DNS provider (often combined with step 1).
Add the domain name to your MOS Admin Center.
Run the verification wizard and add the auto-generated CNAME to your domain's DNS zone.
Validate the domain in the MOS Admin Center.
Start provisioning users with this domain, enable inbound e-mail on this domain, etc.

The verfication step is an important piece, because this helps MOS make sure that you're using a domain you're actually in control of. Otherwise, malicious people could sign in and hijack your domain, which would suck. The way Microsoft does this is actually simple and elegant: they generate a unique CNAME record (that looks very much like a GUID), and ask you to add this CNAME record, pointing back to a server under their control, to your zone. This has lots of advantages:

It's pragmatic. If you can add a CNAME record to a zone file, you effectively control the domain.
It avoids the nastiness that can result in WHOIS-based verification and allows people who register domains to continue using proxy companies, hiding their personal info from WHOIS spammers.
It's relatively easy. You simply have to add a simple record to your DNS; if you can't do this (or your DNS hoster can't do it for you), then you have much bigger problems managing your DNS and verifying your DNS domain under MOS is the least of your problems.
It's low-impact. The generated CNAME is highly unlikely to be queried during normal operations by your users; only MOS is likely to be looking for it. It doesn't require you to repoint your MX records or otherwise make major modifications to your infrastructure if all you want to do is start using online SharePoint and Live Meeting.

Note that just because you add a domain to MOS doesn't mean you have to use it for email! That's a separate operation, which is a two-step process of enabling inbound email for that domain and then updating your MX records appropriately.

More on other MOS functionality coming later...big thanks to the event staff for their kind permission for me to blog!

These are not the solutions you're looking for

Devin L. Ganger — Fri, 27 Jun 2008 03:18:47 GMT

As IT professionals, we are more than often prone to fall to the perils of magical thinking. (I'm sure this is a side-effect of being human, which is a pesky and bothersome condition I will have to do something about one of these days.) Magical thinking in this context is when we have not internalized the intricacies of a problem and instead rely on formulas rather than true understanding to come up with solutions.

At one ISP I used to work at, we had a glorious reclaimed piece of technology, an Auspex NS-5500 file server. Every now and then on reboot, this old beast of a machine would fail to boot up; the cure was to open the cover over the drive cage and give it a good swift whack. We all assumed that this was because one of the drive connectors was a bit loose, but when our "magic" fix failed to work one night I discovered that it was in fact because one of the screws holding things in place was missing, allowing the drive bay to sag just a tiny bit. It was this tiny bit of sag that put just enough stress on the connector for drive 0. Had we actually opened the case up earlier, we'd have been able to solve the problem -- and prevent a year of whacking the server.

All too often, I see magical thinking in the field of security. Case in point: I recently heard about a gentleman who has a client that is requesting ETRN support be added back to Exchange 2007, either natively or through an add-on. They want to deploy the Edge role in their DMZ, have it queue up mail for the internal organization, and then have their Hub Transports (in the internal protected network) initiate a connection out to de-queue the messages using the ETRN SMTP extension. The reason they want this is that they've done due diligence and read some very thorough documents about computer network zones and have come to the conclusion that all network connections must be initiated from the most secure network. This, they say, removes the threat of malware taking over the Edge server in the DMZ and allowing an attacker to use it as a launching point to the protected network.

Now, the recommendation for connections to be initiated from a more secure network to a less secure network is a good general baseline to follow when it makes sense. However, it is not realistic in all cases (if we followed this to the letter, nobody would be able to receive e-mail from external senders except through random polling of Internet SMTP hosts, which is not at all scalable). This is doubly true if you don't understand how the underlying protocols work. Case in point: ETRN, defined by RFC 1985, "SMTP Service Extension for Remote Message Queue Starting". Quoting from section 3, "The Remote Queue Processing Declaration service extension" (emphasis added):

To save money, many small companies want to only maintain transient connections to their service providers. In addition, there are some situations where the client sites depend on their mail arriving quickly, so forcing the queues on the server belonging to their service provider may be more desirable than waiting for the retry timeout to occur.

Both of these situations could currently be fixed using the TURN command defined in [1], if it were not for a large security loophole in the TURN command. As it stands, the TURN command will reverse the direction of the SMTP connection and assume that the remote host is being honest about what its name is. The security loophole is that there is no documented stipulation for checking the authenticity of the remote host name, as given in the HELO or EHLO command. As such, most SMTP and ESMTP implementations do not implement the TURN command to avoid this security loophole.

This has been addressed in the design of the ETRN command. This extended turn command was written with the points in the first paragraph in mind, yet paying attention to the problems that currently exist with the TURN command. The security loophole is avoided by asking the server to start a new connection aimed at the specified client.

See the problem? ETRN was not designed to solve a security problem; it was designed to solve a financial problem back in days when always-on bandwidth was a lot more expensive and most ISPs metered traffic. It masquerades as solving a security problem only because it's designed to avoid a loophole in an insecure and exploitable feature. As a result, ETRN won't solve the problem these people want it to solve; all it does is tell the system in the DMZ to initiate a new connection to the Hub Transport servers. It doesn't reuse the existing connection initiated by the Hub Transport servers. They can't use a firewall rule to block outgoing access from the Edge to the Hub Transport and be safe, because they'll cut off all incoming traffic.

However, let us for a moment assume that it did work the way they wanted it to: my Hub Transport initiates an outbound SMTP session to the Edge. In this session, HT is the SMTP client, ET is the SMTP server. As soon as HT issues the ETRN command, they still have to swap roles -- HT is now using the SMTP server code paths, while the ET is using the SMTP client code paths. Any theoretical vulnerabilities that are in the HT SMTP implementation are still going to be there, still exposed to the message traffic about to be sent down the connection, still open to exploitation.

This is the magical thinking: firewalls and a DMZ will protect my traffic. This is not true; firewalls and networks zones are two components of a complete security plan. Neither firewalls nor network zones can protect legitimate traffic, nor are they designed to; they are designed to allow you to designate which traffic is legitimate. If you want to secure that traffic, you need to turn to other measures.

Tech-Talk: Making Backups Cool with DPM

Devin L. Ganger — Wed, 18 Jun 2008 17:55:50 GMT

While I was at the Tech-Ed NA IT Pro conference last week, Jason Buffington and I took the chance to invade the Tech-Ed Online fishbowl studio and record a quick Tech-Talk on using DPM. You can now view it online on the Tech-Ed IT Pro page and the Library page, or stream it directly. Now that Tech-Ed's over, maybe we'll both find the time to be on Xbox Live at the same time so we can continue our discussion in Call of Duty 4...

Revised guidance on protecting Exchange with DPM 2007

Devin L. Ganger — Wed, 04 Jun 2008 19:36:11 GMT

Just a quick note to let you all know that the Protecting Exchange Server with DPM 2007 white paper is available for download from Microsoft. This is the same white paper I worked on for them last year, but freshly revised to include more guidance around mailbox-level recovery.

I'll be giving a talk around this topic next week at Tech-Ed (IT Pro) in Orlando, session number MGT369. Hope to see you there! (Yes, this is the same talk I did at Exchange Connections in Orlando and in MMS in Vegas a month ago; it seems to be a popular session!)

Three random links make a post

Devin L. Ganger — Mon, 02 Jun 2008 21:28:15 GMT

...so I'll throw in a fourth for good measure. Rather than try to write a full-length post about each of these, I'm just going to give you a quick bullet list:

Want to get the MAPI client or CDO libraries for Exchange 2007, or for Vista and Windows Server 2008? Wait no more: Microsoft Exchange Server MAPI Client and Collaboration Data Objects 1.2.1 is up on Microsoft Downloads.
Microsoft has done another cool thing: the Sysinternals tools are now available live from the Web. If you just need a specific tool, throw in the executable name to the URL and run it.
If you're trying to test a VSS writer, how do you do it? Start by downloading the VSS 7.2 SDK, which contains the vshadow.exe and BETest utilities. Optionally, you can also download the third-party utility Hobocopy.

One last quick tidbit: Exchange 2007 and Outlook Anywhere scalability whitepaper

Devin L. Ganger — Sat, 10 May 2008 00:07:05 GMT

A lot of you may have missed this: Microsoft just released a new white paper for Exchange, Outlook Anywhere Scalability with Outlook 2007, Outlook 2003, and Exchange 2007. This paper should give you some detailed guidance goodness on scaling your CAS servers, and also talks about the port exhaustion issues that lead to upper scalability limits.

A certificate roundup

Devin L. Ganger — Fri, 09 May 2008 23:55:07 GMT

Certificates are one of the biggest issues I keep hearing about with Exchange and OCS, and apparently I'm not the only one. Fellow MVP Michael B. Smith has recently posted two blog articles on certs: how to use SAN certificates with ISA 2006 and other certificate limitations. However, he's got a couple of points on the second article that I'm confused about:

According to this announcement on the Windows Mobile team blog, Windows Mobile 6.0 and up do in fact support wildcard certificates.
The first point he makes is also head-scratcher, because I've also heard this was an issue, but I'd also recently heard of a workaround for it:
1. In Outlook, go to the properties for your Exchange account (Tools, Account Settings, select your Exchange account and click Change) and click More Settings.
2. On the Connection tab, click Exchange Proxy Settings.
3. Look for the field Only connect to proxy servers that have this principal name in their certificate and make sure it's checked (you may need to check the Connect using SSL only checkbox first).
4. The value in this field should normally be set to msstd:server.external.fqdn, the FQDN the server is known as from the outside and that is the subject name of the certificate. So if my certificate was issued for 3Sharp, it would be msstd:mail.3sharp.com. To use this with a wildcard certificate issued to *.3sharp.com, this value would need to be set to msstd:*.3sharp.com.
  
  Let's try a diagram to make the point:

I'm doing more checking, trying to figure out what the deal is here; in the meantime, if you've got operational experience with either of these issues, please let me know.

At any rate, there's some more interesting factoids on certificates I've picked up:

If you want to use a certificate with the Exchange 2007 UM role, you need to have a certificate on the machine whose subject name matches the server's AD/DNS FQDN. It seems that you can't enable a certificate for the UM service using the Enable-ExchangeCertificate cmdlet if this does not match. Note that you can do this for other services, such as those hosted by the CAS role; the cmdlet performs different name checks on the certificate based on the services (SMTP, POP3, IMAP, HTTP, and UM) that you are enabling.
I've said it before, but it needs to be repeated: if you're not using the default self-signed certificate, simply use the Enable-ExchangeCertificate cmdlet to move all services to one or more additional certificates. Do not delete the default certificate; although in most cases Exchange will simply recreate it when the appropriate service is restarted, you can cause subtle errors that will take a while to figure out.
Learn more about certificate usage in Exchange in Creating a Certificate or Certificate Request for TLS.
And learn more about the Enable-ExchangeCertificate cmdlet.

More later!

A DPM roundup

Devin L. Ganger — Fri, 02 May 2008 20:06:22 GMT

This was a big travel week for me; I got the privilege of speaking about protecting Exchange with DPM 2007 at both Exchange Connections (in Orlando) and Microsoft Management Summit (in Las Vegas). The session had a good response at both shows, and there's clearly a lot of buzz going around about DPM. I've gotten some good questions which I'll list here and update as I get answers.

Q: Does DPM protect message tracking logs on an Exchange mailbox server?
A: Very good question. My gut instinct is "No" but I need to confirm that. I'll post the confirmation in a separate blog article when I get an answer back.
Q: Is there any good guidance on sizing a DPM installation?
A: Yes. First see the Data Protection Manager 2007 Storage Calculator (currently only supports the Exchange workload), then see this third-party deconstruction. Note that the second post was written against an earlier release of the calculator, so is in need of some updating, but it's still a good read.
Q: What kind of overhead does DPM incur?
A: I have to admit that I don't remember the specifics of this question (this is why I strongly encourage folks to email their questions to me, as is the case with the following question -- thanks!); all I have is a cryptic note "CPU overhead" on my notepad. So, I'm going to assume that we're talking about the overhead of the protection agent on a protected server. And my answer to that is: Very good question; I need to get some specifics.
Q: From e-mail: "Yesterday during MMS at the Advanced Exchange protection session you mentioned that you had created a white paper on getting DPM working with IBM’s TSM product. If you have a link to this I would be very grateful as I have not been able to find it currently and I am wanting to ensure that they way I have it set up and kind of working is the same way that someone else has been able to get it working."
A: Unfortunately, I must have been unclear, for which I apologize. 3Sharp did work with Microsoft during the DPM 2006 timeframe to create several white papers on how to integrate DPM with several backup products: Commvault QiNetix, Symantec Backup Exec, Yosemite Backup, and Windows Backup. Unfortunately, Tivoli wasn't one of them, and I'm not aware of any current guidance that gives a complete end-to-end picture of integrating TSM with DPM 2007. However, the Backup of DPM Servers section in the DPM Operations Guide should be a good starting place.
Q: Why can't I use DPM 2007 to recover to the Recovery Storage Group on Exchange 2003 servers, only on Exchange 2007 servers?
A: Another great question, which I'm querying to find the answer to.
Q: If I can use DPM 2007 to do document-level recovery in SharePoint, why can't I recover mailboxes or even messages in Exchange without having to use the RSG (for Exchange 2007)or ExMerge (for Exchange 2003)?
A: There are two parts of this answer, but they both are based on the same premise: DPM does not use "privileged" information on the internals of other Microsoft applications it protects. When recovering documents from a SharePoint replica, DPM doesn't directly reach into the replica database and extract the information. Instead, it recovers the relevant databases to a temporary recovery SharePoint installation (which can be a single server SPS 3.0 install on a virtual machine, even if you're recovering data from MOSS 2007) and then finds the relevant documents using SharePoint's HTTP interfaces. With Exchange, the principle is the same; we recover the mailbox database to a parallel location (the RSG in Exchange 2007; a network folder in Exchange 2003) and then use the Exchange native tools to extract and import the relevant information. Trying to do direct restores of mailboxes or messages into a production database would involve going beyond the existing Exchange APIs. Personally, as an Exchange MVP I hope that Microsoft works on expanding those interfaces to make this sort of thing easier for all third-party vendors, but until they do, DPM plays by Exchange's rules.
Q: You mentioned coming updates to DPM. Where can I find more info on that?
A: Jason Buffington of Microsoft has you covered with this webcast.

That's a good start for now; catch you all later!

Greetings from Orlando!

Devin L. Ganger — Mon, 28 Apr 2008 13:53:24 GMT

I'm posting from a break between sessions at Exchange Connections in Orlando, FL. I just had a good session on protecting Exchange with DPM -- thanks to everyone who attended and gave lots of good feedback.

Next up -- a session on DCAR with Exchange, and then Exchange 2007 update best practices.

The weather is actually the best I've ever seen here -- not too hot, with a nice breeze, so the humidity isn't overwhelming. However, the A/C is up full in the room I'm presenting, so I'm glad the speaker shirts are long-sleeved.

More later!

Setting Exchange 2007 Unified Messaging codecs on a per-user basis? Genius!

Devin L. Ganger — Wed, 23 Apr 2008 22:06:19 GMT

I was completely floored to discover, via Paul, that you can control which codec the UM role uses to record voicemails on a per-user basis. This is seriously cool stuff, and if you can't see why quite yet, let me offer the following scenarios for you:

Most common: you have multiple users who have non-Windows Mobile devices that don't support the WMA codec, but still want to be able to listen to their voicemail on their devices. The GSM and G.711 PCM Linear codecs may be more widely supported. For example, on an EAS-aware iPhone will Apple also roll in support for recognizing UM voicemails? If they do, will they support the WMA codec? Now, in theory, they don't have to.
Also common: you have multiple users who use a non-Windows based client. (Paul already calls out one example, those of us who use Entourage.) This would be just as valuable, though, for people who are using some IMAP or POP3 client on a Linux/BSD/Solaris box.
Not so common, but possible: you have a specific need to automatically process voicemails in an automated fashion and need to use either the GSM or G.711 PCM linear codecs instead of being able to support WMA. Switching one or two mailboxes over keeps the entire Exchange storage system from suffering the increase in voicemail file size that would result.

Okay, so these are slightly lame scenarios, but I'm sure there's more out there that I can't see.