[jhollington]

Originally Posted by brong

That's been something we've been very careful about.

...and it shows

I've moved my mail back and forth between FastMail and other services, including my own Dovecot server, enough times that I've come to appreciate the intricacies of good IMAP (many of my "migration tools" over the years have been my own hacked-together Python and Perl scripts).

Then there's the inscrutable way that the iOS Mail app handles IMAP sessions and UID FETCH, especially in larger mailboxes, and what it does to the sort order of messages if they weren't filed into the folders chronologically in the first place (as often happens when moving large chunks of messages or reorganizing things). On my own server, I went so far as to actually run a script to reset the mtimes on the underlying mbox files and then force a regeneration of all UIDs so that everything would be nice and chronological

Forget non-standard server IMAP implementations. Some ideas of how to build IMAP clients are even nuttier.

There's also debate over whether we will allow multiple exactly identical messages in the same mailbox. Same bytes. If we don't, it would be implemented as immediately expunging the old copy when you added a new one, because that's safest against IMAP clients which do something like move-to-current-folder through some bug, and proceed to EXPUNGE the old UIDs.

While I'm obviously not up on all of the technical details, I feel like de-duplication of that nature would probably solve a lot of problems. I know that your own IMAP migration tools can handle this well enough, but there are a lot of folks who just think it's a good idea to grab a handful of messages from another IMAP mail provider, or even a local folder, and drag and drop them into a FastMail IMAP folder as a means of "migration" and of course that's going to result in duplicates if something's not done to deal with it. Ironically, Gmail seemed to deal with this in some way — at least from a user-facing perspective, I can't comment on what actually ended up in the database

To make matters even more complicated, I know that some mail clients — Apple Mail for macOS comes to mind — have actually started filtering out duplicate messages in the UI, regardless of what exists on the IMAP server or in the underlying store. Doesn't mean the messages won't be there, but the end user will never see them, which can lead to all sorts of other confusion when doing back-end comparisons of message counts.

Certainly, it's going to be a bit complex allowing both UID view and JMAP ID view of the same messages, because we'll have to merge the flags somehow - if you set a flag via JMAP we probably want to set it on all messages, but if you clear it from ONE of the UIDs via IMAP, do you want to clear it from all? Tricky.

I can see that. Heck, I've run into that scenario with poorly designed IMAP clients when moving large chunks of messages around where there are duplicates that have different flag status, and JMAP doesn't even enter into that.