EmailDiscussions.com  

Go Back   EmailDiscussions.com > Email Service Provider-specific Forums > FastMail Forum
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
Stay in touch wirelessly

FastMail Forum All posts relating to FastMail.FM should go here: suggestions, comments, requests for help, complaints, technical issues etc.

Reply
 
Thread Tools
Old 6 Jan 2017, 07:35 AM   #1
petar
Junior Member
 
Join Date: Jan 2017
Posts: 4
Imported Gmail messages take much less space?

Hi all,

I am new to fastmail and I just finished migrating my Gmail messages to fastmail - about 11000 messages (since 2004) that take over 2 GB, if I believe Google stats. I removed all labels and made visible in IMAP only "All", "Sent" and "Inbox" (which is normally empty). It took few experiments to come to this, but all went well and surprisingly fast, only that all this mail takes a bit over 600 MB in Fastmail, and this is suspicious. The number of messages match more or less (different, but around 11000 - a bit more actually), and reviewing the oldest and the newest they seems to be there, although I can not be REALLY sure for the oldest. As I tried this few times until I make the import to my liking, it wasevery time about this size. I wonder why Gmail may be showing bigger size - I often clean my mail from old semi-spam, and big attachments, but it seemed to be reflected on the spot. Difference that big seems off, although I can not identify something missing.

So to state the question - does anyone has explanation for this - do Gmail inflate used storage somehow (for example count messages twice in some cases), or Fastmail compresses messages by default?
petar is offline   Reply With Quote

Old 6 Jan 2017, 08:17 AM   #2
BritTim
The "e" in e-mail
 
Join Date: May 2003
Location: mostly in Thailand
Posts: 2,437
What an interesting post! While it has never occurred to me before, it suggests that efficiency of storage use must be taken into account, as well as storage quotas.It may suggest that Fastmail's pricing is more competitive than is sometimes claimed.

I believe Fastmail does compress email but, as you say, the discrepancy seems rather large to be explained simply by that. I am wondering if the following might be true:
  • there is a problem with garbage collection in the Google File System under some circumstances; or
  • Google counts duplicate space (taken up for reliability and performance reasons, while Fastmail definitely does not.
I wonder what ideas others might have.
BritTim is offline   Reply With Quote
Old 6 Jan 2017, 09:31 AM   #3
n5bb
Intergalactic Postmaster
 
Join Date: May 2004
Location: Irving, Texas
Posts: 8,118
Arrow Sending large messages to Gmail then deleting them recovers all space

I also find this a fascinating thread! It's hard to imagine what would cause a 3X difference. Binary files are usually sent using base64 encoding, but that only expands the file by a factor of 4/3, and I doubt that FastMail saves attachments as binary files. Maybe Gmail saves several hash tables or other metadata to make it easier to perform searches. But a 3X ratio is huge!

I just sent 5 identical messages with 14.7 MB attachments each from FastMail to Gmail. In FastMail each message size was about 20.25 MB (after base64 encoding), so the total size should be about 101 MB. In Gmail my used space (at the bottom of the page) changed from 1.72 GB to 1.82 GB, so that matches the expected size. I then deleted the 5 messages and permanently deleted them from Trash. My used space went back up to 1.72 GB. So my tests so far don't show anything unusual.

Hmmm ... I think I will try moving messages back and forth using an IMAP email client and see what happens.

Bill
n5bb is offline   Reply With Quote
Old 6 Jan 2017, 01:24 PM   #4
n5bb
Intergalactic Postmaster
 
Join Date: May 2004
Location: Irving, Texas
Posts: 8,118
Arrow Messages moved to Gmail and Fastmail via IMAP client affect memory use identically

I moved about 100 MB of messages to/from Gmail using an IMAP client (Thunderbird), then did the same thing with an IMAP connection to Fastmail.
  • The Gmail memory used changed by 100 MB.
  • The Fastmail memory used also changed by 100 MB. The Fastmail memory value may take a couple of minutes to update, so be patient.
  • The Fastmail memory value at the bottom of the folder tree shows the value in bytes. So my legacy Enhanced account shows a total memory size of 15,728 MB.
  • In Settings>Folders>Quota usage (link at the bottom) the memory size is shown in multiples of M = 1024 X 1024 = 1,024,576 Bytes.
  • This means that my Enhanced account has a total size of 15,000 M X 1.048576 MB/M = 15,728 MB.
So I wasn't able to duplicate your results. Gmail and Fastmail appear to me to show memory size identically. Could your results be due to extra messages hidden in the Gmail account? Gmail is very annoying for me to use due to how they make user of labels rather than folders, especially with an email client. It's easy to delete messages and they appear to be gone but are actually just hidden with labels. I know you tried to take this into account, but some Gmail behavior might have kept some of the message on the Gmail server.

Bill
n5bb is offline   Reply With Quote
Old 6 Jan 2017, 01:40 PM   #5
Terry
The "e" in e-mail
 
Join Date: Jul 2002
Location: VK4
Posts: 2,457
Fastmail may delete duplicates....
Terry is offline   Reply With Quote
Old 6 Jan 2017, 02:04 PM   #6
n5bb
Intergalactic Postmaster
 
Join Date: May 2004
Location: Irving, Texas
Posts: 8,118
That's a good idea, Terry. Gmail and Fastmail both suppress duplicates which arrive at the account in the normal fashion, but (at least for Fastmail) this suppression does not affect IMAP transfer:
http://www.emaildiscussions.com/show...16&postcount=7

Bill
n5bb is offline   Reply With Quote
Old 6 Jan 2017, 08:52 PM   #7
petar
Junior Member
 
Join Date: Jan 2017
Posts: 4
That's what I suspected - that Google duplicates many messages for some reason. The thing is that I have removed all custom labels before importing. Basically I had one label - All (which is visible as Archive in IMAP) that has all the mail, Sent and empty Inbox. I have checked the option to eliminate duplicates on import.
The only explanation is that gmail keeps many duplicates for some reason - probably they were in some label and then moved or something like this - otherwise the total number of messages in "All mail" that I can see in Gmail web interface and the imported messages almost match - around 11000. They are not equal, but since the imported ones are more, I hope nothing is lost
At the end this is just curiosity. Keeping old mail is somewhat weird for me, I think important info should be kept elsewhere and mail should be transactional, but for many years I relied on the Google mentality "don't delete anything" and I am still a bit nervous that I may lose something. I have not needed an old mail for quite a while, however.
petar is offline   Reply With Quote
Old 6 Jan 2017, 10:40 PM   #8
BritTim
The "e" in e-mail
 
Join Date: May 2003
Location: mostly in Thailand
Posts: 2,437
At this point, I would be inclined to use the IMAPSize program to see if the "folder" sizes match up with total used space.
BritTim is offline   Reply With Quote
Old 6 Jan 2017, 11:13 PM   #9
petar
Junior Member
 
Join Date: Jan 2017
Posts: 4
Good suggestion, I will.

Edit: This program is only for Window, but I looked in Trash and here we go - it seems that it is cleaned a lot less often than I thought. After I cleaned it, everything matches. No mystery anymore. Kind of petty

Last edited by petar : 6 Jan 2017 at 11:31 PM.
petar is offline   Reply With Quote
Old 7 Jan 2017, 04:07 AM   #10
BritTim
The "e" in e-mail
 
Join Date: May 2003
Location: mostly in Thailand
Posts: 2,437
Quote:
Originally Posted by petar View Post
Good suggestion, I will.

Edit: This program is only for Window, but I looked in Trash and here we go - it seems that it is cleaned a lot less often than I thought. After I cleaned it, everything matches. No mystery anymore. Kind of petty
I think I ran IMAPsize in the past under Wine. Anyway, good in a way that the mystery is solved, but kind of disappointing that the explanation is so mundane!
BritTim is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT +9. The time now is 09:21 AM.

 

Copyright EmailDiscussions.com 1998-2013. All Rights Reserved. Privacy Policy