Moving Mailboxes Off a Shared Host: imapsync, DNS and the Outlook Autodiscover Trap
How I moved a small business mail domain to my own server: imapsync in Docker, delta syncs, MX/SPF/DKIM cutover, and why Outlook ignored my autoconfig.
A small business I run the infrastructure for had their mail on a cheap shared hoster. The contract was cancelled, the old host kept the mailboxes alive for a little while as a courtesy, and I had to move everything to my own server before that ran out. One main mailbox, one scanner mailbox, a few devices, and one person who lives in Outlook.
The mail copy was the easy part. Outlook was not.
TL;DR: Copy with imapsync in Docker (--user 0 or it can't read the password files), do a full sync, cut DNS over, then run delta syncs for the same mailbox until the old host goes dark. Put autoconfig/autodiscover on the mail domain itself, not on the server's hostname. And when Outlook still says "Something went wrong" while your server logs a successful login: that's Microsoft's cloud, not you. Use classic Outlook.
The target
New box: Ubuntu 24.04 with KeyHelp as the control panel. Domain created as mail-only (the website stays where it is), plus mail.example.com as its own domain with a Let's Encrypt cert so clients connect to a name that matches the certificate. If you run that setup with Postfix, read the SNI texthash fix first, it bit me once already.
Mailboxes created in the panel, passwords generated, written to a root-only file. Nothing special.
Step 1: imapsync, in Docker
There's no imapsync package in Ubuntu 24.04's repos, and I didn't want Perl modules all over the host. The official image works fine:
mkdir -p /root/mail-sync && cd /root/mail-sync
printf '%s' 'old-password' > old.pass
printf '%s' 'new-password' > new.pass
chmod 600 *.pass
docker run --rm --user 0 -v "$PWD":/sync gilleslamiral/imapsync imapsync \
--host1 imap.old-host.example --port1 993 --ssl1 \
--user1 office@example.com --passfile1 /sync/old.pass \
--host2 mail.example.com --port2 993 --ssl2 \
--user2 office@example.com --passfile2 /sync/new.pass \
--automap --dry
Drop --dry once the folder mapping looks right. --automap maps Sent/Trash/Drafts between the different naming schemes, so you don't end up with "Sent" and "Sent Items" next to each other.
I wrapped this in a small sync.sh that loops over a list of mailboxes, with an ONLY= variable to run a single one. You'll want that later.
The main mailbox: 1,616 messages, 298 MiB, 0 errors.
Gotcha: imapsync exits with code 66
First run died right away with exit code 66. The image runs as a non-root user by default, and my password files were 600 root. imapsync couldn't read its passfile and gave up. --user 0 fixes it. The alternative is loosening the file permissions, and I'd rather not do that for passwords.
Gotcha: the old host only had the folders it had
The source mailbox had INBOX, one custom folder and the standard folders. Before the sync I'd assumed there was a big folder tree to move. There wasn't. Check --dry output for the folder list before you promise anyone "all your folders are there".
The scanner mailbox didn't sync at all, the old password I had was wrong. Since it only gets scans and nobody needs old scans, we skipped it. Ask before you spend an hour recovering something nobody cares about.
Step 2: DNS cutover
Lower the TTL a day before, then switch:
example.com. MX 10 mail.example.com.
mail.example.com. A 203.0.113.10
example.com. TXT "v=spf1 mx a:server.example.net -all"
default._domainkey TXT "v=DKIM1; k=rsa; p=MIIBIjANBg..."
_dmarc.example.com. TXT "v=DMARC1; p=none; rua=mailto:dmarc@example.com"
autoconfig.example.com. A 203.0.113.10
autodiscover.example.com. A 203.0.113.10
The website A record stays untouched, mail and web live on different boxes. SPF with -all only after you've checked there's nothing else sending as that domain (contact forms, invoicing tools, the copier). I explain the DMARC part in reading DMARC reports.
Step 3: delta syncs
Mail keeps arriving at the old host until every sender's resolver has picked up the new MX. So after the cutover, run the same command again for the mailbox that matters:
cd /root/mail-sync && ONLY=office@example.com ./sync.sh
imapsync skips messages it already copied (it compares by Message-ID and headers), so a delta run is quick. I keep doing that until the old host shuts down.
Step 4: forwarding without breaking SPF
The main mailbox also had to forward a copy to three people. Plain forwarding means the next server sees your server sending mail "from" the original sender's domain, and a strict SPF there fails it. postsrsd (Sender Rewriting Scheme) rewrites the envelope sender on forward, so SPF checks against your domain instead. KeyHelp had it on already. Keep a copy in the mailbox too, so the forward isn't the only place the mail exists.
Step 5: Outlook (the actual work)
The mail copy was boring. Outlook is where the time went.
Gotcha: the panel's autoconfig answers with the server hostname
Mail clients look up autoconfig.example.com/mail/config-v1.1.xml (Thunderbird) or autodiscover.example.com/autodiscover/autodiscover.xml (Outlook). KeyHelp ships a catch-all for both. Nice, except that catch-all always hands out the server's own hostname, not mail.example.com. Clients then connect to a name that has nothing to do with the customer's domain.
Fix: create autoconfig.example.com and autodiscover.example.com as real domains in the panel, with Let's Encrypt, and put your own files in the docroot:
<!-- mail/config-v1.1.xml -->
<clientConfig version="1.1">
<emailProvider id="example.com">
<domain>example.com</domain>
<incomingServer type="imap">
<hostname>mail.example.com</hostname>
<port>993</port>
<socketType>SSL</socketType>
<username>%EMAILADDRESS%</username>
<authentication>password-cleartext</authentication>
</incomingServer>
<outgoingServer type="smtp">
<hostname>mail.example.com</hostname>
<port>587</port>
<socketType>STARTTLS</socketType>
<username>%EMAILADDRESS%</username>
<authentication>password-cleartext</authentication>
</outgoingServer>
</emailProvider>
</clientConfig>
Plus a small autodiscover.php behind an .htaccess rewrite that returns the same servers in Outlook's XML format. This works because per-domain vhosts come before the catch-all in Apache. Test with curl https://autoconfig.example.com/mail/config-v1.1.xml.
Gotcha: Outlook asks Microsoft, not you
Here's what I didn't expect. Current Outlook first asks Microsoft's own autodetect service. That service makes its guess from the MX host and caches it for hours. So while I was fixing autodiscover, Outlook never even asked my server. It kept getting the old answer from Microsoft's cache.
Gotcha: "Something went wrong" in the new Outlook
The new Outlook for Windows doesn't talk IMAP from your PC. Microsoft's cloud logs in to your server and syncs the mailbox for you. So you get this:
- my server logs a successful IMAP login
- Outlook shows "Something went wrong" and some text about app passwords
The login worked. Whatever broke happened in Microsoft's sync service after that, and there's nothing on your server to fix. I stopped fighting it and set the account up in classic Outlook with manual settings (IMAP 993 SSL, SMTP 587 STARTTLS). Worked right away. The one error we did get in classic Outlook was simply an old saved password.
How to tell who's actually connecting
Always look at the mail log instead of guessing from client error messages:
grep 'office@example.com' /var/log/mail.log | grep -E 'Login|auth' | tail -20
The rip= field is the remote IP. If it's a Microsoft range and not the customer's office IP, it's the new Outlook's cloud sync talking, not the PC. Once I knew that, half the "Outlook can't connect" reports made sense.
What I'd do the same way next time
- imapsync in Docker, one script,
ONLY=for single mailboxes - full sync before DNS, delta syncs after
- autoconfig/autodiscover as real per-domain vhosts from day one
- classic Outlook with manual settings, skip the wizard
- mail log first, client error messages second
If you're on a Plesk box with a dead license and mail is the last thing keeping you there, this is the way out.