Microsoft Exchange weirdness kills my morning

09 February 2009 in Internet by Stilgherrian | 9 comments

I usually don’t write about geeky problems. However I just lost the entire morning troubleshooting a weird situation with Microsoft Exchange 2003 and I’d like to understand it. If I asked you to read this, read on…

The problem was that Exchange’s POP3 connector was saying it had retrieved a user’s email and delivered it into their mailbox. However when we looked in the mailbox, the email wasn’t there. Nor was it in any of the “undeliverable” queues. Nor were there any error messages. I think I’ve solved it — or at least figured out a workaround — but I’d like to understand Exchange’s behaviour here. So here goes…

This is a relatively-new server running Microsoft Small Business Server 2003, fully patched, with (of course) Exchange 2003. It does not accept email directly via SMTP, except on the local LAN; all email from external addresses is brought in using the POP3 Connector from mailboxes in a shared hosting account on a Linux server which is under my control.

The primary email domain is maindomain.com.au. Each user has a base email address of the form jsmith@maindomain.com.au, and some users have john@maindomain.com.au added and marke as their primary address. Some users also have addresses in other domains, such as jsmith@additionaldomain.com.au or john@additionaldomain.com.au, which I’d added using the “Manage Users” tool in SBS’s GUI.

All this was working OK, and still is.

However on the weekend I added another additional domain, let’s call it problematic.com.au, and two more users. For these users, I wanted jbrown@problematic.com.au to be their primary address in Outlook, so I added the SMTP addresses to their account in the usual way, set them as primary addresses, and then added the relevant POP3 Connectors. POP3 connector downloads the email OK, and the logs would show that it was successfully delivered into the mailbox jbrown@problematic.com.au. However it wasn’t. Email to jbrown@maindomain.com.au failed silently in the same way.

Here’s the weirdness…

If I set the user’s primary email address to jbrown@maindomain.com.au, then both jbrown@maindomain.com.au and jbrown@problematic.com.au start working. Change it back and they fail again.

Also, the second user, “Sue Smith”, was sue@problematic.com.au and the primary address and ssmith@maindomain.com.au as the add-on. Both failed. If I flip them so ssmith@maindomain.com.au is the primary email, then ssmith@maindomain.com.au starts working but sue@problematic.com.au still fails.

This is presumably because the underlying Windows login is ssmith and not sue. But why should this matter if the email address is listed in the user’s account as an SMTP address?

It looks like email is only accepted if the primary email address is within the server’s primary domain, but not if an address in another domain is set as the primary — even though both addresses are attached to the user. And, as it happens, even when problematic.com.au is added to the server’s default recipient policy.

Am I understanding this correctly?

And, the final scary question… Since there’s nothing in the delivery or undeliverable queues, has the inbound email which didn’t arrive been lost forever?

Possibly related posts

Tags: exchange, microsoft, sbs, smtp

9 comments

Comments feed for this article

Trackback link: http://stilgherrian.com/internet/microsoft-exchange-weirdness-kills-my-morning/trackback/

Tatham Oddie on 09 February 2009 at 3:18 pm

It’s not really a solution as much as a suggestion, but have you checked out http://www.quantumsoftware.com.au/ProductsServices/ExchangeConnector.aspx ?

It’s a third party POP3 connector developed here in Sydney that absolutely flogs the built in connector. Much more reliable to start with and much better logging to help diagnose any issues. Also, local phone support.

Reply
Stephen Edgar on 09 February 2009 at 3:21 pm

Answer (Scroll to the bottom past the ad’s) http://twurl.nl/gnccph

Reply
Stephen Edgar on 09 February 2009 at 3:23 pm

Edit: Open Google.com.au and type this into the search box without the quotes:-

“SBS 2003 Internet Connection Wizard POP 3 Connector”

Click on the first result and scroll to the bottom of the page.

Reply
Stilgherrian on 09 February 2009 at 3:48 pm

I think I’ve managed to work out how to deal with this. Running the “Configure E-mail and Internet Connection Wizard” seems to ensure that all of the add-on email addresses are properly communicated through to the POP3 Connector. I’ll update this and respond to other comments when I finish testing.

Reply
Stilgherrian on 10 February 2009 at 5:18 am
@Tatham Oddie: Thanks for pointing out the third party Exchange Connector. I’ll definitely trial that with future clients.

One of the key features — apart from any increased reliability — is the ability to collect POP3 email more frequently than Microsoft’s POP3 Connectors’ hard-wired 15 minutes. I’ve always wondered why Outlook can be set to collect mail every 60 seconds but Exchange only one-fifteenth the speed. It’s almost as if they wanted to give the impression that “Internet email” is slow… Almost.

@Stephen Edgar: Although that link didn’t have the answer visible for me, for some reason, that was the solution. THANK YOU SO MUCH!

This is what I did:
- In “Server Management”, right-click on each user, go to the “E-mail Addresses” tab and added the new SMTP address(es).
- Run the “Configure E-mail and Internet Connection Wizard”, changing nothing but just hitting “Next” and “Next” until Finish” and then letting it rebuild the email settings.
- Re-start the POP3 Connector service.
The key was running the “Configure E-mail and Internet Connection Wizard”. I don’t know whether restarting the POP3 Connector was essential, but I didn’t want to break the voodoo.

Unfortunately the email which arrived in the interim is lost forever. It turns out that Microsoft turned off the “badmail” folder by default in Small Business Server 2003 Service Pack 1 — though I haven’t got the reference handy for that, I closed the browser tab. I’m guessing that’s because people’s servers fell over when the “badmail” folder eventually filled up — so to stop it filling up, you just turn off that feature.

I’ll see if I can get Microsoft to comment on this, because it seems a silly way to go.

Fortunately the two users affected use a low volume of email. When I asked what to do about the “lost” mail I was told, “Fuck it.”

Again, thank you.
Reply
Stephen Edgar on 10 February 2009 at 8:49 am

I have been bitten by not running the “Configure E-mail and Internet Connection Wizard” on SBS before and wondered why things broke when I did it manually.

Voodoo shit in SBS and as you just run it going ‘next, next, etc, finish’ it must tweak something funky somewhere.

And yes that first link did not work when going directly to Expert’s Exchange — but if you Google your problem and then click through the Google link to the Experts Exchange site you get the answers at the bottom of the page unhidden. (Nice Hack )

A quick reference I found:-

http://support.microsoft.com/kb/885685 though when compared to

http://web.archive.org/web/20050214153649/http://support.microsoft.com/kb/885685

‘Note ‘The Badmail folder is disabled in Microsoft Exchange Server 2003 Service Pack 1
(SP1).’

Very different document revisions….

Cheers,

Stephen

Reply
Cassie ST on 10 February 2009 at 12:46 pm

*Waves* Glad you got it sorted Stil.. Filters tripped me up. (Doh!)

Reply
Nick Hodge on 16 February 2009 at 4:10 pm

Stilgherrian

As per your request to escalate to Microsoft; I did just that. Here is the response:

1. Not 100% sure but I think this link may provide more information on why the manual configuration failed: http://support.microsoft.com/kb/325464. The solution provided in his blog explains to use the Configure E-mail and Internet Connection Wizard which seems to be the recommended way to set up SBS POP3 connections.

2. The articles are actually the same kb article, one is an old revision posted in Dec 2004 which is revision 1.2 the other is the current updated version of the same article dated Dec 2007 which is revision 2.4. Regardless, the information in both regarding badmail is the same. The following article written by the Exchange dev team explains the decision to disable badmail by default: http://msexchangeteam.com/archive/2004/07/20/188856.aspx

3. SBS 2008 includes Exchange 2007 and the POP3 connector is still included, however please reference the following statement from the SBS2008 documentation:

“The Exchange 2007 POP3 connector is available. However, it is recommended that you consider migrating to a hosting solution where you can receive e-mail using SMTP. The POP3 Connector is designed to provide new installs with the ability to have e-mail available right away using Exchange and thus moving the burden of mail checking from clients, but it is not designed or intended to be a long term solution.”

Reply
Stilgherrian on 16 February 2009 at 5:03 pm

@Nick Hodge: Thanks. I wasn’t aware that the POP3 Connector isn’t seen as a long-term tool — and presumably neither had the people who’d set up other Windows servers I’ve taken over. They saw it as the “right” tool to use for a small business on a dynamic IP address.

Indeed, I saw it as a sensible choice from a security and maintenance cost viewpoint: not having an SMTP port open to all IP addresses significantly reduces the inbound traffic and immediately rules out a whole class of potential attacks.

Reply