(e)Mail Insecurity

DPM 2007 Rollup packages now available

Devin L. Ganger — Thu, 10 Jul 2008 02:34:31 GMT

While I was away on vacation last week, Microsoft finally released the DPM 2007 Rollup packages to Microsoft Downloads. (I blame Jason Buffington; I'm sure he waited until I was out of office.) There are both x86 and x64 packages; both require you to download three separate files.

In addition to various bug fixes, this rollup (also known as a "feature pack") provides the following new functionality:

Official support for protecting Windows Server 2008 servers (and supported applications, such as Exchange Server 2007, running on Windows 2008), including protecting the system state.
You get support for backing up clustered Virtual Server 2005 R2 SP1 environments. Before, the cluster itself was not seen as a cluster by DPM, and depending on your configuration you may have needed to do some funky scripting.
Better tape handling. You can now share tape libraries between multiple DPM servers, reducing the cost of long-term tape retention and allowing better utilization of high-end tape libraries. You can also put multiple protection groups on a single tape; DPM 2007 RTM would start a new tape as it began writing each protection group, even if the previous tape was not fully used. This could get expensive.

I haven't yet been able to confirm whether the cleaning tape bug Tim noted has been fixed in this update, but I suspect not.

Applying this update is a four-step process:

Install the main DPM update (DataProtectionManager2007-KB949779.exe)on your DPM servers.
Install the SQL Server update (SqlPrep-KB949779.msp) on the machine hosting the SQL Server database for DPM. In a default install, this is the same machine that is your DPM server.
Update the agents on your protected servers to version 2.0.8107.0. You can push them out through the console or manually run the .msp update package on your protected machines (using any supported push mechanism). You will need to restart the protected machines for the new agent version to take effect.
Update the DPM Management Shell update (DPMManagementShell2007-KB949779.msp) on all of your DPM management stations (including the DPM servers themselves).

Although the official instructions give the update steps in the previous order, I have run all three udpates on my lab DPM servers before updating the agents on my protected servers, and as long as Microsoft doesn't say that's not supported, that's the way I'd recommend doing it -- that way, all of your PowerShell tasks are using the updates even if you don't have all the protection agents pushed out yet.

Hyper-V in the hizzouse!

Devin L. Ganger — Fri, 27 Jun 2008 03:28:20 GMT

Everyone's being so coy in the Windows blogosphere today. "As you may have heard..." Heck with that; this is wicked cool. Hyper-V has Released To Manufacturing ... and is already available for download. As the link explains, it'll start coming down the Windows Update pipe July 8th. If you don't want your Windows Server 2008 machine to be updated yet, don't be blindly accepting updates.

Why wouldn't you want to get it first thing?

You're running a previous version of Hyper-V. If so, be aware that upgrading your VMs is not automatic. It's not a horrible process, but it will take some time. You have to manually export each VM, remove the VMs from the server, upgrade the server, re-import the VMs, then update the Integration Services. The more VMs you have, the more time this will take.
You're running some software that is not yet compatible with Hyper-V RTM but works with an earlier build. In this case, you want to wait until that software has a patch available.

I fit into both categories. I think I'm going to wait until I'm back from vacation to do it.

Oh, yes, just because Hyper-V is now RTM doesn't mean that you can go run to install Exchange 2007 on it in production. See Scott Schnoll's post for more info.

These are not the solutions you're looking for

Devin L. Ganger — Fri, 27 Jun 2008 03:18:47 GMT

As IT professionals, we are more than often prone to fall to the perils of magical thinking. (I'm sure this is a side-effect of being human, which is a pesky and bothersome condition I will have to do something about one of these days.) Magical thinking in this context is when we have not internalized the intricacies of a problem and instead rely on formulas rather than true understanding to come up with solutions.

At one ISP I used to work at, we had a glorious reclaimed piece of technology, an Auspex NS-5500 file server. Every now and then on reboot, this old beast of a machine would fail to boot up; the cure was to open the cover over the drive cage and give it a good swift whack. We all assumed that this was because one of the drive connectors was a bit loose, but when our "magic" fix failed to work one night I discovered that it was in fact because one of the screws holding things in place was missing, allowing the drive bay to sag just a tiny bit. It was this tiny bit of sag that put just enough stress on the connector for drive 0. Had we actually opened the case up earlier, we'd have been able to solve the problem -- and prevent a year of whacking the server.

All too often, I see magical thinking in the field of security. Case in point: I recently heard about a gentleman who has a client that is requesting ETRN support be added back to Exchange 2007, either natively or through an add-on. They want to deploy the Edge role in their DMZ, have it queue up mail for the internal organization, and then have their Hub Transports (in the internal protected network) initiate a connection out to de-queue the messages using the ETRN SMTP extension. The reason they want this is that they've done due diligence and read some very thorough documents about computer network zones and have come to the conclusion that all network connections must be initiated from the most secure network. This, they say, removes the threat of malware taking over the Edge server in the DMZ and allowing an attacker to use it as a launching point to the protected network.

Now, the recommendation for connections to be initiated from a more secure network to a less secure network is a good general baseline to follow when it makes sense. However, it is not realistic in all cases (if we followed this to the letter, nobody would be able to receive e-mail from external senders except through random polling of Internet SMTP hosts, which is not at all scalable). This is doubly true if you don't understand how the underlying protocols work. Case in point: ETRN, defined by RFC 1985, "SMTP Service Extension for Remote Message Queue Starting". Quoting from section 3, "The Remote Queue Processing Declaration service extension" (emphasis added):

To save money, many small companies want to only maintain transient connections to their service providers. In addition, there are some situations where the client sites depend on their mail arriving quickly, so forcing the queues on the server belonging to their service provider may be more desirable than waiting for the retry timeout to occur.

Both of these situations could currently be fixed using the TURN command defined in [1], if it were not for a large security loophole in the TURN command. As it stands, the TURN command will reverse the direction of the SMTP connection and assume that the remote host is being honest about what its name is. The security loophole is that there is no documented stipulation for checking the authenticity of the remote host name, as given in the HELO or EHLO command. As such, most SMTP and ESMTP implementations do not implement the TURN command to avoid this security loophole.

This has been addressed in the design of the ETRN command. This extended turn command was written with the points in the first paragraph in mind, yet paying attention to the problems that currently exist with the TURN command. The security loophole is avoided by asking the server to start a new connection aimed at the specified client.

See the problem? ETRN was not designed to solve a security problem; it was designed to solve a financial problem back in days when always-on bandwidth was a lot more expensive and most ISPs metered traffic. It masquerades as solving a security problem only because it's designed to avoid a loophole in an insecure and exploitable feature. As a result, ETRN won't solve the problem these people want it to solve; all it does is tell the system in the DMZ to initiate a new connection to the Hub Transport servers. It doesn't reuse the existing connection initiated by the Hub Transport servers. They can't use a firewall rule to block outgoing access from the Edge to the Hub Transport and be safe, because they'll cut off all incoming traffic.

However, let us for a moment assume that it did work the way they wanted it to: my Hub Transport initiates an outbound SMTP session to the Edge. In this session, HT is the SMTP client, ET is the SMTP server. As soon as HT issues the ETRN command, they still have to swap roles -- HT is now using the SMTP server code paths, while the ET is using the SMTP client code paths. Any theoretical vulnerabilities that are in the HT SMTP implementation are still going to be there, still exposed to the message traffic about to be sent down the connection, still open to exploitation.

This is the magical thinking: firewalls and a DMZ will protect my traffic. This is not true; firewalls and networks zones are two components of a complete security plan. Neither firewalls nor network zones can protect legitimate traffic, nor are they designed to; they are designed to allow you to designate which traffic is legitimate. If you want to secure that traffic, you need to turn to other measures.

masteringdpm.com back online

Devin L. Ganger — Mon, 23 Jun 2008 15:26:49 GMT

Things got hairy enough last week that I forgot to post, but my hosting provider got the problem sorted out and the website is back online.

masteringdpm.com temporarily down

Devin L. Ganger — Wed, 18 Jun 2008 18:43:45 GMT

If you've tried to get to masteringdpm.com in the past couple of days, you may have gotten a cryptic error message instead of a site with DPM goodness. I'm working with my hosting provider to get it put back up ASAP and will post again once it's back up.

Tech-Talk: Making Backups Cool with DPM

Devin L. Ganger — Wed, 18 Jun 2008 17:55:50 GMT

While I was at the Tech-Ed NA IT Pro conference last week, Jason Buffington and I took the chance to invade the Tech-Ed Online fishbowl studio and record a quick Tech-Talk on using DPM. You can now view it online on the Tech-Ed IT Pro page and the Library page, or stream it directly. Now that Tech-Ed's over, maybe we'll both find the time to be on Xbox Live at the same time so we can continue our discussion in Call of Duty 4...

Welcome, Mike Rand!

Devin L. Ganger — Tue, 17 Jun 2008 19:46:23 GMT

Just a quick shout-out to fellow 3Sharpie Mike Rand, who just posted his first post to the 3Sharp blog site last week. Mike's a super-smart developer here with mad SharePoint skills; I can't imagine why he hasn't blogged sooner than this, but I hope to see him posting more frequently! He's also pretty good at foosball.

Updated Exchange Developer Roadmap

Devin L. Ganger — Tue, 17 Jun 2008 19:43:45 GMT

To reinforce yesterday's post about Exchange Web Services (EWS), I wanted to draw your attention to the Exchange Developer Roadmap posted on May 22 2008 on the Exchange API-spotting blog.

There shouldn't really be any surprises here, but there were a couple of items I wanted to highlight. First:

Given this commitment to Web services and our goal of making Exchange Web Services the richest developer interface for Exchange... (emphasis added)

Here's a preview of some of the functionality that we plan to add to the next release of Exchange Web Services:

Access to Folder Associated Items (FAI) and read/write access to user settings (Devin: this page in the MAPI reference indicates that FAIs are things like views and forms. I believe that this also fixes a known quirk of EWS that keeps you from creating Outlook-visible search folders that use certain property paths. I believe this also gives access to server-side rules, if they're not already accessible through a separate part of the API.)

Management of Personal Distribution Lists (Devin: very cool.)

Throttling capabilities that give Exchange administrators control over system resource consumption (Devin: this will be very nice for helping keep poorly written applications from taking down the Exchange servers.)

A powerful and easy-to-use server-to-server authentication model to enable building portals and enterprise mash-ups (Devin: let's hope this can ease some of the pain of building Exchange-aware SharePoint sites, at least those that don't require direct access to private mailbox content.)

An easy-to-use Microsoft .NET API that fully wraps the Web service calls, which makes Web service development even easier (Devin: I'll be interested in seeing how this stacks up against third-party offerings like the Independentsoft EWS client offering.)

Then they go on to list the APIs that will get removed (Exchange WebDAV, Store Events, CDO 3.0/CDOEx, and ExOLEDB) and moved to "extended support" (Exchange Server MAPI Client, CDO 1.2.1). Don't get too excited by the MAPI client -- it's not what you think:

Provides server applications a MAPI runtime for accessing Exchange.

Note: This is not the Outlook MAPI Client library that is included with Outlook.

and

Outlook's Exchange MAPI Store provider, available in the Outlook MAPI Client library can also be used to access an Exchange mailbox or public folder.

If you're going to start writing Exchange-aware applications, you should probably start looking at EWS first for future compatibility. If you're trying to support Exchange 2003 at the same time...good luck.

A .NET add-on for working with Exchange Web Services

Devin L. Ganger — Mon, 16 Jun 2008 18:10:20 GMT

I just got word that Independentsoft has come out with a beta version of an EWS client API for the .NET Framework and .NET Compact Framework. I've not looked at it yet, but I'm particularly hopeful about having a good way to work with EWS from Windows Mobile devices.

Exchange Web Services (EWS), introduced in Exchange 2007 and enhanced in Exchange 2007 SP1, is Microsoft preferred interface for all future programmatic reach into the Exchange store. While EWS is a Web service, it can be pretty complicated to work with. Luckily, we've done some work with EWS here at 3Sharp; Paul's been presenting some developer training sessions on EWS in partnership with Microsoft. We've found that Inside Microsoft Exchange Server 2007 Web Services has been a valuable reference on EWS.

One of the challenges for EWS development is that the schema and object model is pretty complex when compared with the typical Web service, enough so that you need to use special Visual Studio proxy classes when you use .NET to work with EWS. This, by the way, is very likely the cause of the compatibility issue I found between EWS and SharePoint Designer -- Designer's proxy classes aren't the EWS-aware ones.

3Sharp, Podcasting, and You

Devin L. Ganger — Mon, 09 Jun 2008 18:25:11 GMT

The talented people at 3Sharp are one of the best reasons to work here. Our Platforms Group is just one piece of the pie here; we've got some top-tier development talent who can make SharePoint stand up and dance. Those guys down the hall have been working hard on a little surprise they like to call the Podcasting Kit for SharePoint, which Microsoft has just released on Codeplex as indicated in their press release. 3Sharpies John Peltonen, David Gerhardt, and Paul Robichaux are also blogging about it, so if you’re interested, check them out.

I've been hearing bits and pieces, but last week I got to sit down and take a good look at what they're doing. Wow. This is some cool stuff that is going to make sharing podcasts, video talks, and other knowledge sharing content a lot easier. I can't wait until I can start using it; I've already lined up some content that I can put up and I'm already thinking of some more I can do.

All purchases should be this easy

Devin L. Ganger — Mon, 09 Jun 2008 17:45:18 GMT

If you haven't seen me in person recently, you may not realize I'm a heretic. Yes, that's right -- I use an Apple 15" MacBook Pro with Vista as my laptop. It took some jiggling to get it all working -- an upgrade to Leopard (OS X 10.5) for the final release of BootCamp, an upgrade to Vista SP1, and finding a stable version of the Atheros wireless drivers -- but it's now reliable and fast.

There are some downsides to this particular laptop. It's only gives me 2GB of RAM, which means that I can't run a typical VM configuration (DC, DPM, Exchange) and still have enough power to run PowerPoint like I could under XP. The battery life is okay but not great; I run out on long flights.

I'm off to Tech-Ed this week, so I stopped by the Apple store in Bellevue Square Sunday to pick up a spare battery for the flight. I've had bad experiences at this store in the past; I don't give off the right vibe(or maybe I just look light a tightwad) and can't get seem to get the attention of the staff. I took a chance, though, and walked in the store.

This time, my customer service experience was great. I caught the eye of Associate 1; although he was busy with another customer, he called for help; I didn't even see him do it. A minute later, Associate 2 walks up to me. "I understand you're looking for a 15" MacBook Pro battery." Pleasantly shocked, I followed him over to the appropriate shelf and soon had the battery in hand. "Is there anything else I can help you with, or are you ready to check out?"

If you've not been into an Apple store recently, they're doing something absolutely sweet. Each customer service associate has a hip-mounted scanner/cardreader. They scan your merchandise on the spot, take and run your credit card, and ask you for an email address to send the receipt to. Boom -- it's all done, your card is charged, and you don't have to stand in line at the counter unless you're doing cash or check. This is a great concept I'd love to see other stores use. My receipt hit my Exchange account (and thus my Windows Mobile phone) as I was walking out of the store.

I love living in the future.

Revised guidance on protecting Exchange with DPM 2007

Devin L. Ganger — Wed, 04 Jun 2008 19:36:11 GMT

Just a quick note to let you all know that the Protecting Exchange Server with DPM 2007 white paper is available for download from Microsoft. This is the same white paper I worked on for them last year, but freshly revised to include more guidance around mailbox-level recovery.

I'll be giving a talk around this topic next week at Tech-Ed (IT Pro) in Orlando, session number MGT369. Hope to see you there! (Yes, this is the same talk I did at Exchange Connections in Orlando and in MMS in Vegas a month ago; it seems to be a popular session!)

Hyper-V RC1 available

Devin L. Ganger — Tue, 03 Jun 2008 02:01:48 GMT

This is pretty cool -- I didn't even notice this at first! Hyper-V RC1 is now available for download through the Microsoft Download center or through Windows Update as an optional update. One of the nice changes here is that you now install the Hyper-V Integration Services on Windows 2008 guest machines the same way as any other operating system (before, you'd have to install the Hyper-V patch itself as a separate action).

That would be why my Windows Server 2008 machine wanted an extra reboot this afternoon...

Three random links make a post

Devin L. Ganger — Mon, 02 Jun 2008 21:28:15 GMT

...so I'll throw in a fourth for good measure. Rather than try to write a full-length post about each of these, I'm just going to give you a quick bullet list:

Want to get the MAPI client or CDO libraries for Exchange 2007, or for Vista and Windows Server 2008? Wait no more: Microsoft Exchange Server MAPI Client and Collaboration Data Objects 1.2.1 is up on Microsoft Downloads.
Microsoft has done another cool thing: the Sysinternals tools are now available live from the Web. If you just need a specific tool, throw in the executable name to the URL and run it.
If you're trying to test a VSS writer, how do you do it? Start by downloading the VSS 7.2 SDK, which contains the vshadow.exe and BETest utilities. Optionally, you can also download the third-party utility Hobocopy.

One last quick tidbit: Exchange 2007 and Outlook Anywhere scalability whitepaper

Devin L. Ganger — Sat, 10 May 2008 00:07:05 GMT

A lot of you may have missed this: Microsoft just released a new white paper for Exchange, Outlook Anywhere Scalability with Outlook 2007, Outlook 2003, and Exchange 2007. This paper should give you some detailed guidance goodness on scaling your CAS servers, and also talks about the port exhaustion issues that lead to upper scalability limits.

A certificate roundup

Devin L. Ganger — Fri, 09 May 2008 23:55:07 GMT

Certificates are one of the biggest issues I keep hearing about with Exchange and OCS, and apparently I'm not the only one. Fellow MVP Michael B. Smith has recently posted two blog articles on certs: how to use SAN certificates with ISA 2006 and other certificate limitations. However, he's got a couple of points on the second article that I'm confused about:

According to this announcement on the Windows Mobile team blog, Windows Mobile 6.0 and up do in fact support wildcard certificates.
The first point he makes is also head-scratcher, because I've also heard this was an issue, but I'd also recently heard of a workaround for it:
1. In Outlook, go to the properties for your Exchange account (Tools, Account Settings, select your Exchange account and click Change) and click More Settings.
2. On the Connection tab, click Exchange Proxy Settings.
3. Look for the field Only connect to proxy servers that have this principal name in their certificate and make sure it's checked (you may need to check the Connect using SSL only checkbox first).
4. The value in this field should normally be set to msstd:server.external.fqdn, the FQDN the server is known as from the outside and that is the subject name of the certificate. So if my certificate was issued for 3Sharp, it would be msstd:mail.3sharp.com. To use this with a wildcard certificate issued to *.3sharp.com, this value would need to be set to msstd:*.3sharp.com.
  
  Let's try a diagram to make the point:

I'm doing more checking, trying to figure out what the deal is here; in the meantime, if you've got operational experience with either of these issues, please let me know.

At any rate, there's some more interesting factoids on certificates I've picked up:

If you want to use a certificate with the Exchange 2007 UM role, you need to have a certificate on the machine whose subject name matches the server's AD/DNS FQDN. It seems that you can't enable a certificate for the UM service using the Enable-ExchangeCertificate cmdlet if this does not match. Note that you can do this for other services, such as those hosted by the CAS role; the cmdlet performs different name checks on the certificate based on the services (SMTP, POP3, IMAP, HTTP, and UM) that you are enabling.
I've said it before, but it needs to be repeated: if you're not using the default self-signed certificate, simply use the Enable-ExchangeCertificate cmdlet to move all services to one or more additional certificates. Do not delete the default certificate; although in most cases Exchange will simply recreate it when the appropriate service is restarted, you can cause subtle errors that will take a while to figure out.
Learn more about certificate usage in Exchange in Creating a Certificate or Certificate Request for TLS.
And learn more about the Enable-ExchangeCertificate cmdlet.

More later!

Doing UC in the Pacific Northwest

Devin L. Ganger — Tue, 06 May 2008 17:19:34 GMT

I've been sitting on a cool announcement for several days now, and I'm happy that it's now time to announce it.

I've been working with a group of people to get a new user group for Unified Communications (UC) put together here in the Pacific Northwest. While all of us are here in the Puget Sound area, our goal is to put in place a framework to empower a variety of events and meetings all throughout the region, not just based here in Seattle. Rather than be a typical boring user group with a jawbreaking acronym (PNWUCUG, which we do use), we're defining ourselves as people who do UC. This gives us a simpler name -- We do UC, hosted at ucdoers.org.

From our website:

We are the Pacific Northwest Unified Communications User Group (PNWUCUG) and we have a passion for UC. If you are one of the following, you could be one of us:

IT professionals in the Pacific Northwest who design, deploy, or manage Exchange Server, Live Communications Server, and Office Communications Server systems.
Developers who write or maintain solutions that integrate, extend, or provide UC capabilities to Exchange Server, Live Communications Server, and Office Communications Server and clients.
Industry experts with a recognized expertise in UC.
Hobbyists who are exploring Microsoft-based UC solutions.

One thing that's important for me to clarify -- my vision of this user group (which is echoed by the other folks who are getting it off the ground) is that it exists to support all Exchange, LCS, and OCS users, not just people running 2007 and doing the VoIP stuff. We may have a focus on UC, but that's mainly to align ourselves with the direction Microsoft is taking these products. If you're using Exchange, we want you to participate; we want to make sure we have content for you.

So, if this sounds like goodness to you, head on over to the blog for the announcement of our May 28th kick-off meeting at The Parlor Billiards & Spirits in Bellevue, WA. For those of you who can't be there in person, we're even going to have a Live Meeting feed for you -- how cool is that?

Post-Conference report

Devin L. Ganger — Fri, 02 May 2008 20:16:15 GMT

As I typically do, I'm posting links to my slide decks for the presentations I just finished giving. I apologize to the Connections folks; I was supposed to get this done Monday afternoon or Tuesday and got ambushed by a travel-induced migraine.

Orlando was nice this time of year; not too hot, so the humidity slipped under the radar. It was nice to see a bunch of familiar faces and meet some new ones, and I was very pleased with the attendance at all of my sessions. Doing all three sessions back-to-back is definitely a drain, but the conference organizers helped out a lot by keeping me in the same room for all of them, and had I stayed for a couple of days I'd definitely have had the . And I have apparently finally beaten my notorious string of demo failures; my demo DPM environment (provided by Jason Buffington of Microsoft, thank you Jason) worked quite nicely.

For the MMS folks, I can't put my deck up directly; you'll need to get it from the MMS CommNet or wait for your attendee DVD to show up. Las Vegas is still completely over the top; the Venetian was opulent and provided a nice venue. For some reason, the casino didn't seem nearly as intrusive as it could have been (and is in other venues). I am, however, glad I had new shoes -- my feet didn't hurt from all the walking. For the flight home, I picked up 21: Bringing Down the House - Movie Tie-In: The Inside Story of Six M.I.T. Students Who Took Vegas for Millions at the airport and read it cover-to-cover; a great story told well.

A DPM roundup

Devin L. Ganger — Fri, 02 May 2008 20:06:22 GMT

This was a big travel week for me; I got the privilege of speaking about protecting Exchange with DPM 2007 at both Exchange Connections (in Orlando) and Microsoft Management Summit (in Las Vegas). The session had a good response at both shows, and there's clearly a lot of buzz going around about DPM. I've gotten some good questions which I'll list here and update as I get answers.

Q: Does DPM protect message tracking logs on an Exchange mailbox server?
A: Very good question. My gut instinct is "No" but I need to confirm that. I'll post the confirmation in a separate blog article when I get an answer back.
Q: Is there any good guidance on sizing a DPM installation?
A: Yes. First see the Data Protection Manager 2007 Storage Calculator (currently only supports the Exchange workload), then see this third-party deconstruction. Note that the second post was written against an earlier release of the calculator, so is in need of some updating, but it's still a good read.
Q: What kind of overhead does DPM incur?
A: I have to admit that I don't remember the specifics of this question (this is why I strongly encourage folks to email their questions to me, as is the case with the following question -- thanks!); all I have is a cryptic note "CPU overhead" on my notepad. So, I'm going to assume that we're talking about the overhead of the protection agent on a protected server. And my answer to that is: Very good question; I need to get some specifics.
Q: From e-mail: "Yesterday during MMS at the Advanced Exchange protection session you mentioned that you had created a white paper on getting DPM working with IBM’s TSM product. If you have a link to this I would be very grateful as I have not been able to find it currently and I am wanting to ensure that they way I have it set up and kind of working is the same way that someone else has been able to get it working."
A: Unfortunately, I must have been unclear, for which I apologize. 3Sharp did work with Microsoft during the DPM 2006 timeframe to create several white papers on how to integrate DPM with several backup products: Commvault QiNetix, Symantec Backup Exec, Yosemite Backup, and Windows Backup. Unfortunately, Tivoli wasn't one of them, and I'm not aware of any current guidance that gives a complete end-to-end picture of integrating TSM with DPM 2007. However, the Backup of DPM Servers section in the DPM Operations Guide should be a good starting place.
Q: Why can't I use DPM 2007 to recover to the Recovery Storage Group on Exchange 2003 servers, only on Exchange 2007 servers?
A: Another great question, which I'm querying to find the answer to.
Q: If I can use DPM 2007 to do document-level recovery in SharePoint, why can't I recover mailboxes or even messages in Exchange without having to use the RSG (for Exchange 2007)or ExMerge (for Exchange 2003)?
A: There are two parts of this answer, but they both are based on the same premise: DPM does not use "privileged" information on the internals of other Microsoft applications it protects. When recovering documents from a SharePoint replica, DPM doesn't directly reach into the replica database and extract the information. Instead, it recovers the relevant databases to a temporary recovery SharePoint installation (which can be a single server SPS 3.0 install on a virtual machine, even if you're recovering data from MOSS 2007) and then finds the relevant documents using SharePoint's HTTP interfaces. With Exchange, the principle is the same; we recover the mailbox database to a parallel location (the RSG in Exchange 2007; a network folder in Exchange 2003) and then use the Exchange native tools to extract and import the relevant information. Trying to do direct restores of mailboxes or messages into a production database would involve going beyond the existing Exchange APIs. Personally, as an Exchange MVP I hope that Microsoft works on expanding those interfaces to make this sort of thing easier for all third-party vendors, but until they do, DPM plays by Exchange's rules.
Q: You mentioned coming updates to DPM. Where can I find more info on that?
A: Jason Buffington of Microsoft has you covered with this webcast.

That's a good start for now; catch you all later!

Greetings from Orlando!

Devin L. Ganger — Mon, 28 Apr 2008 13:53:24 GMT

I'm posting from a break between sessions at Exchange Connections in Orlando, FL. I just had a good session on protecting Exchange with DPM -- thanks to everyone who attended and gave lots of good feedback.

Next up -- a session on DCAR with Exchange, and then Exchange 2007 update best practices.

The weather is actually the best I've ever seen here -- not too hot, with a nice breeze, so the humidity isn't overwhelming. However, the A/C is up full in the room I'm presenting, so I'm glad the speaker shirts are long-sleeved.

More later!

Setting Exchange 2007 Unified Messaging codecs on a per-user basis? Genius!

Devin L. Ganger — Wed, 23 Apr 2008 22:06:19 GMT

I was completely floored to discover, via Paul, that you can control which codec the UM role uses to record voicemails on a per-user basis. This is seriously cool stuff, and if you can't see why quite yet, let me offer the following scenarios for you:

Most common: you have multiple users who have non-Windows Mobile devices that don't support the WMA codec, but still want to be able to listen to their voicemail on their devices. The GSM and G.711 PCM Linear codecs may be more widely supported. For example, on an EAS-aware iPhone will Apple also roll in support for recognizing UM voicemails? If they do, will they support the WMA codec? Now, in theory, they don't have to.
Also common: you have multiple users who use a non-Windows based client. (Paul already calls out one example, those of us who use Entourage.) This would be just as valuable, though, for people who are using some IMAP or POP3 client on a Linux/BSD/Solaris box.
Not so common, but possible: you have a specific need to automatically process voicemails in an automated fashion and need to use either the GSM or G.711 PCM linear codecs instead of being able to support WMA. Switching one or two mailboxes over keeps the entire Exchange storage system from suffering the increase in voicemail file size that would result.

Okay, so these are slightly lame scenarios, but I'm sure there's more out there that I can't see.

Security and the OCS 2007 A/V Edge role

Devin L. Ganger — Fri, 11 Apr 2008 18:33:42 GMT

When people start digging into the specifics of the A/V Edge role in OCS 2007, they usually have a strong and immediate knee-jerk reaction something along the lines of, "No way!" (Mine was, "Oh, heck no!") This reaction is usually caused by learning one or more of the following deployment requirements:

Public IP address. The A/V Edge server needs to have a publicly routable IP address. This address must be publicly routable; you can't fudge it by giving it an IP address in a private range (10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16) and do any sort of NAT to it. 1:1 NAT or static NAT mapping won't do the trick here. You can and should have a firewall between it and the Internet, but it can't be doing any address translation.
Dual-homed. The A/V Edge server cannot be separated from the internal OCS servers by NAT. Therefore, if you're using a private address range and NAT in your internal network, you have to give the A/V Edge server a second network interface and IP address on routable, non-NAT address range. (Note, however, it doesn't have to be the same address range as the internal network, simply on an address range that is directly routable without NAT.)
20,0002 external ports. The external (publicly routable) interface needs to have the following ports opened to the Internet: UDP 3478, TCP 443, UDP 50,000-59,999, and TCP 50,000-59,999. Security people immediately look at the need to have 10,000 dynamic TCP ports and 10,000 dynamic UDP ports and have their head asplode in sheer instinctive security reaction.

I've personally reacted to all three of these requirements; I've yet to talk to a security-conscious IT professional new to OCS who hasn't. So what on Earth is Microsoft doing putting these requirements in place? Have they completely lost it about security?

In a word, no.

There are good reasons why these requirements are in place. Rather than go over them myself, however, let me simply direct you to this excellent post on the OCS team blog. If you have any questions, post them there and tell 'em I sent you. Note that to post questions on their blog, you need to first join their Community Server site. This is painless and easy; simply click the Join link in the upper right-hand corner, pick a username and password, provide your email address, and you're ready to go.

Exchange protocol documentation now available

Devin L. Ganger — Fri, 11 Apr 2008 02:21:31 GMT

Per the announcement on Tuesday (08 Apr), Microsoft has released a lot of new documentation for various Exchange and Outlook-Exchange protocols. This is some cool stuff -- just check out the list of what's available. However, as the web site warns, it's preliminary documentation. If you don't believe them, when you download the files (available in PDF format) the big fat "PRELIMINARY" watermark (in very bold font) will help remind you.[1]

I can already hear some of you out there: "So Microsoft released documentation on obscure or unimportant Exchange protocols. Big deal. I bet they've saved all the good stuff for licensing!" Well, I'm not going to deny that this is a complete set of documentation for every Exchange protocol you might ever want to know about -- after all, Microsoft is a company who believes in the value of intellectual property. They've kinda built a business plan around it, and it's both foolish and naive to somehow assume that they're just going to toss all of that overboard overnight. It's not even reasonable to expect them to completely abandon that position; it's an arguable proposition that Open Source principles work best in conjunction with an IP scheme that permits open licensing when the developers feel invested in doing so, alongside more restrictive licensing schemes. But that's a religious argument for another day.

This will be a long post. I'm going to split it into three sections: Appetizers, Main Course, and What's Missing.

Section 1: Appetizers

First, we have some housekeeping and overview documents and protocols:

Name	Description
MS-CAB	Cabinet File Format
MS-MCI	MCI Compression and Decompression
MS-OXDOCO	Outlook-Exchange Protocol Document Roadmap
MS-OXGLOS	Office Exchange Protocols Master Glossary
MS-OXPROTO	Office Exchange Protocols Overview
MS-OXREF	Office Exchange Protocols Master Reference
MS-PATCH	LZX DELTA Compression and Decompression

You may have noticed that these documents include a few things that aren't strictly Exchange or Outlook-specific, such as the CAB file format and various compression protocols. Just remember that the Exchange protocol documentation is part of a wider set of Interoperability Principles, and so it depends on technologies that are part of the more generic set of Windows technologies.

Section 2: Main Course

Okay, with roadmaps and preliminaries out of the way, let's take a look at the meat:

Name	Description
MS-NSPI	Name Service Provider Interface (NSPI) Protocol Specification
MS-OXABREF	Address Book Name Service Provider Interface (NSPI) Referral Protocol Specification
MS-OXBBODY	Best Body Retrieval Protocol Specification
MS-OXCDATA	Data Structures Protocol Specification
MS-OXCETF	Enriched Text Format (ETF) Message Body Conversion Protocol Specification
MS-OXCFOLD	Folder Object Protocol Specification
MS-OXCFXICS	Bulk Data Transfer Protocol Specification
MS-OXCICAL	iCalendar to Appointment Object Conversion Protocol Specification
MS-OXCMAIL	RFC2822 and MIME to E-mail Object Conversion Protocol Specification
MS-OXCMSG	Message and Attachment Object Protocol Specification
MS-OXCNOTIF	Core Notifications Protocol Specification
MS-OXCPERM	Exchange Access and Operation Permissions Specification
MS-OXCPRPT	Property and Stream Object Protocol Specification
MS-OXCROPS	Remote Operations (ROP) List and Encoding Protocol Specification
MS-OXCRPC	Wire Format Protocol Specification
MS-OXCSPAM	Spam Confidence Level, Allow and Block Lists Protocol Specification
MS-OXCSTOR	Store Object Protocol Specification
MS-OXCSYNC	Mailbox Synchronization Protocol Specification
MS-OXCTABL	Table Object Protocol Specification
MS-OXDISCO	Autodiscover HTTP Service Protocol Specification
MS-OXDSCLI	Autodiscover Publishing and Lookup Protocol Specification
MS-OXIMAP4	Internet Message Access Protocol Version 4 (IMAP4) Extensions Specification
MS-OXLDAP	Lightweight Directory Access Protocol (LDAP) Version 3 Extensions Specification
MS-OXMSG	.MSG File Format Specification
MS-OXMVMBX	Mailbox Migration Protocol Specification
MS-OXOAB	Offline Address Book (OAB) Format and Schema Protocol Specification
MS-OXOABK	Address Book Object Protocol Specification
MS-OXOABKT	Address Book User Interface Templates Protocol Specification
MS-OXOCAL	Appointment and Meeting Object Protocol Specification
MS-OXOCFG	Configuration Information Protocol Specification
MS-OXOCNTC	Contact Object Protocol Specification
MS-OXODLGT	Delegate Access Configuration Protocol Specification
MS-OXODOC	Document Object Protocol Specification
MS-OXOFLAG	Informational Flagging Protocol Specification
MS-OXOJRNL	Journal Object Protocol Specification
MS-OXOMSG	E-mail Object Protocol Specification
MS-OXONOTE	Note Object Protocol Specification
MS-OXOPFFB	Public Folder Based Free/Busy Protocol Specification
MS-OXOPOST	Post Object Protocol Specification
MS-OXORMDR	Reminder Settings Protocol Specification
MS-OXORMMS	Rights-Managed E-mail Object Protocol Specification
MS-OXORSS	RSS Object Protocol Specification
MS-OXORULE	E-mail Rules Protocol Specification
MS-OXOSFLD	Special Folders Protocol Specification
MS-OXOSMIME	S/MIME E-mail Object Protocol Specification
MS-OXOSMMS	SMS and MMS Object Protocol Specification
MS-OXOSRCH	Search Folder List Configuration Protocol Specification
MS-OXOTASK	Task-Related Objects Protocol Specification
MS-OXOUM	Voice Mail and Fax Objects Protocol Specification
MS-OXPFOAB	Offline Address Book (OAB) Public Folder Retrieval Protocol Specification
MS-OXPHISH	Phishing Warning Protocol Specification
MS-OXPOP3	Post Office Protocol Version 3 (POP3) Extensions Specification
MS-OXPROPS	Office Exchange Protocols Master Property List Specification
MS-OXPSVAL	E-mail Postmark Validation Protocol Specification
MS-OXRTFCP	Rich Text Format (RTF) Compression Protocol Specification
MS-OXRTFEX	Rich Text Format (RTF) Extensions Specification
MS-OXSHARE	Sharing Message Object Protocol Specification
MS-OXSMTP	Simple Mail Transfer Protocol (STMP) Mail Submission Extensions Specification
MS-OXTNEF	Transport Neutral Encapsulation Format (TNEF) Protocol Specification
MS-OXWAVLS	Availability Web Service Protocol Specification
MS-OXWOAB	Offline Address Book (OAB) Retrieval Protocol Specification
MS-OXWOOF	Out of Office (OOF) Web Service Protocol Specification
MS-OXWUMS	Voice Mail Settings Web Service Protocol Specification
MS-XJRNL	Journal Record Message Format Protocol Specification
MS-XLOGIN	SMTP Protocol AUTH LOGIN Extension Specification
MS-XWDVSEC	Web Distributed Authoring and Versioning (WebDAV) Protocol Security Descriptor Extensions Specification

On first glance, that's an impressive list. NSPIs, S/MIME, SMTP and POP3 extensions, RTF extensions, TNEF -- the list goes on. There's a lot of seriously crunchy material here. The question of the moment, though, is "just how detailed is all this documentation?"

Good question.

I haven't had time to look through it all in a lot of detail. To be honest, I suspect that a lot of it is in areas that I wouldn't be able to catch any glaring omissions or discrepancies (sorry, readers, I'm just not up on the latest specs for RTF). However, I did take a quick look through MS-XLOGIN, "SMTP Protocol AUTH LOGIN Extension Specification"[2], since I'm reasonably familiar with SMTP.

Let me skip to the chase: yup, this is preliminary work. On whole, it does a good job of documenting the flow of the LOGIN extension (which people have already mostly figured out how it works through years of careful protocol analysis). The most complicated part of it is that you're using Base64 to encode the credentials being passed -- not rocket science. However, there are some gaps in this straightforward documentation:

Nowhere did I find any guidance on what the user and passwords challenges are supposed to be computed on (only that they are to be Base64 encoded). This makes it more difficult to properly code a LOGIN implementation.
The samples they gave look like valid Base64, but according to my quick conversion tests in PowerShell, they aren't. I can't get any of the sample values to match what they should. Again, this means I can't work backwards to get the missing data.

I really hope this is the kind of stuff they're going to fix between this release and the final release, because without it, this documentation isn't nearly as useful as it could be. Some would even accuse it of being provided merely to give the appearance of interoperability while still keeping enough implementation details close to the chest to keep it from really happening. I, however, subscribe to the philosophy that one should never initially ascribe to malice what can be explained through other possibilities -- and I've done enough work on these sorts of projects to know that getting the right level of detail in a document like this is far from a no-brainer, especially if you're dealing with contractors or are having to generate the documentation after the fact. (I don't know that either of these possibilities are involved, but I'm guessing.)

Section 3: What's Missing

There are three obvious protocols missing in all of the above: MAPI, MAPI over RPC over HTTP, and Exchange ActiveSync. I can hear the screams now...but this is where I go back to the point that Microsoft still makes money from intellectual property. Microsoft's Web site offers a searchable IP Catalog that shows you exactly which protocols they offer for a licensing fee, and both MAPI (aka the Outlook Exchange Transport Protocol) and Exchange ActiveSync are on it, as well as several other important protocols for Unified Communications. Microsoft is under no obligation to make every single protocol available for free -- but the fact that they're finding value in doing it with the above protocols is pretty cool and interesting. [3]

[1] If the watermark bugs you, try seeing if your PDF client will allow you to view or print the document without annotations. Using Foxit Reader, I was able to make the watermark go away and actually read some of the text it obscured.

[2] SMTP Protocol? Seriously? Is that like PIN number or ATM machine? Attention, Microsoft technical writers: P stands for Protocol.

[3] You're free to speculate on what value they get, but not here, please. That's another religious discussion.

There's no service like Web service

Devin L. Ganger — Sat, 29 Mar 2008 01:21:19 GMT

One of the cool things about Exchange 2007 is the new Web service interface into the store. In theory, having mailboxes and contents exposed via Web services makes it a lot easier for developers and casual dabblers to use Web service-aware tools to interact with Exchange content.

Two weeks ago, I wanted to perform a quick experiment by seeing if I could use Exchange Web Services (EWS) in a SharePoint page to make an always up-to-date extension list for our office. Now, I know this information is stored in Active Directory as attributes on the User objects, but I didn't see a quick, easy way to configure a SharePoint web part to perform an LDAP or AD query. Instead, I opened up SharePoint Designer and pointed it toward our EWS instance, and what I found surprised me.

Does anyone out there in reader land have any clue why SharePoint Designer insists that an EWS instance isn't "a valid description of an XML Web service"?

https://exchange.server.fqdn/ews/Services.wsdl

I can browse to it manually, enter my credentials, and get a bunch of XML that sure looks like valid WSDL -- but SharePoint Designer's integrated WSDL parser can't seem to make heads or tails of it. I could easily consume other types of Web services, and looking at their WSDL, it looks like it's making use of a lot fewer XML namespaces; their XML structure seems quite a bit simpler than Exchange is generating.

I tried contacting the official SharePoint team blog and was basically told, "Go away, kid. Call support." I've not had a lot of spare time recently to pursue this, but I'm pursuing some other avenues to see if I can't get to the bottom of this. Stay tuned!

A connection I hadn't noted before

Devin L. Ganger — Thu, 13 Mar 2008 20:55:16 GMT

Archiving 101's post today made a connection I hadn't thought about before.