Google groups

[mayzie@fastmail]

I subscribe to a couple of different google groups - one uses the address "X.groups.io" and the other uses "X.googlegroups.com" with the individual sender's names shown in the From line.

Fastmail's spam system regularly sends emails from these groups to the Spam folder. I've been clicking on "not spam" for about a year, and it doesn't seem to help - they still go to the Spam folder about half the time. I've added rules sending emails from these addresses directly to the Inbox, and that also doesn't seem to make a difference.

How can I ensure mail from Google Groups gets directly into my Inbox?
Thanks

[TenFour]

I'm not that familiar with Google Groups, but I would guess that the Sender Guidelines might be a good place to start. https://support.google.com/a/answer/...535517360&rd=1

https://support.google.com/a/answer/...71662674688-NA

[mayzie@fastmail]

Thanks ... but I'm talking about receiving mail from a google group, not sending. Your link seems to be about sending emails.

[TenFour]

But if the senders are not following best practices their messages would go into your Spam. Like some emails I get from certain vendors--go right into Spam all the time because of some settings at their end. Nothing I can do about it.

[mayzie@fastmail]

Oh I see! Of course, I have no control over how Google send their messages.

But Google groups are so common and widely used, I really would have thought Fastmail would have a way to recognize them as not spam.

[odedp]

Have you tried adding these addresses to your contacts?

[JeremyNicoll]

Originally Posted by mayzie@fastmail

I subscribe to a couple of different google groups - one uses the address "X.groups.io"

Groups.io is nothing to do with google - it's a completely separate host of discussion groups - see: https://groups.io/ ... whose usage went up a lot when Yahoo stopped hosting groups. Lots of those moved to groups.io.

Fastmail support should be able to tell you what it is about the emails coming from groups.io's servers which makes FM's systems think they are spam.

It might be something to do with the individual owners (people) for the discussion groups (at Groups.io) that you're a member of; maybe there's options they can select that influence how Groups.io then sends mails out to subscribers? I'm a member of 22 discussion groups that they host, and - for me - mails from none of those groups end up in my spam folder. It's true that a lot of those groups are highly technical computing ones where spammers are pretty much unknown; possibly because people can't join the groups until they've been seen only to post sensible things. If your discussion groups allow anyone to join and then to post anything at all that might mean posts from them have a worse potential-spam score.


It might also be that you've set your automatic Fastmail spam-processing rules too aggressively.

[mayzie@fastmail]

Originally Posted by odedp

Have you tried adding these addresses to your contacts?

Thanks, yes, I've added the main email address to my contacts - but I think the problem is that individual emails show the sender's address in the From line, and the lists have hundreds of subscribers so I can't get them all into my contacts.

[mayzie@fastmail]

Originally Posted by JeremyNicoll

Groups.io is nothing to do with google - it's a completely separate host of discussion groups - see: https://groups.io/ ... whose usage went up a lot when Yahoo stopped hosting groups. Lots of those moved to groups.io.

Thanks for clarifying about Groups.io! Don't know why I thought they were part of Google.

Yes, it may be related to individual group settings I guess (but I'd like to have some idea what I'd be asking them to change before I approached the owners). I do have my FM spam protection set to "standard", the lowest option besides "Off".

I'll try FM support and see if they can help.

[n5bb]

I subscribe to several groups.io groups. In the last 30 days I have received 92 such messages at my Fastmail account, and not a single one was marked as spam. My spam threshold is set to 5.0, which is the Standard spam setting threshold.

I recommend that you look at the groups.io messages in your Spam folder.

• In the list of messages in your Spam folder you should find a spam badge with a number (such as 5.5) in the lower right corner of the message in the list, below the date (or possibly below the message size). This number shows the spam score assigned to that message by the Fastmail spam protection system.
• For a Standard spam protection setting the spam filtering system will only move incoming new messages to your Spam folder if the spam score is 5.0 or greater. If the spam score is below 5.0, that means that message made it to your Spam folder in some other manner, such as by you marking the message as spam or by a rule which moved the message to the Spam folder.
• So my first question is whether all of the groups.io messages (or other messages which seem to be improperly placed there) have a spam badge score of 5.0 or greater? If messages with a lower spam score are in your Spam folder, then we need to investigate how they are getting there.

Next I recommend that you select one of the groups.io messages with a spam score of 5.0 or greater so you are reading that message. Then open the Actions menu on the far right side (under the date) and select Show raw message. This will open up a new browser window which shows the raw content of that complete message. I know that this screen will look very complex, but we only need to examine a few lines which start with X-. Those X- header lines have special Fastmail-specific information which tell us the internal state of the system when the message was processed. Look for these lines:

• X-Spam-Score = This value of this number should be the same spam score you saw in the message listing spam badge icon.
• X-Spam-SourceIP = This shows information about the server which sent this message to Fastmail.
• X-Spam-hits = This shows a list of the tags which make up the spam score. Some of these may have a positive value (such as BAYES_50 0.8 or DCC_CHECK 1.1) while others may have a negative value (such as ME_SC_NH -0.001). The algebraic sum of all of those X-Spam-Hits tags becomes the X-Spam-score value. Near the end of X-Spam-hits you may see BAYES_USED user, which indicates that the BAYES spam score is affected by previous user choices you made by marking messages as spam or not-spam.

What you should be looking for is the reason for the high spam score. Is it a high BAYES value (indicating things like message content and spam classification choices you made) or some other tag? I know this seems complicated if you haven’t looked at the full headers below, but you can discover a lot of details about what is going on with those messages by examining the X-Spam headers.

Bill

[JeremyNicoll]

Originally Posted by JeremyNicoll

... where spammers are pretty much unknown; possibly because people can't join the groups until they've been seen only to post sensible things.

It occurred to me that this sounds impossible - how can someone prove they post sensibly without joining?. What I meant is that in some groups new members (who've joined only in the sense that they've been added to the list of members and can read other peoples' posts) are 'moderated' for their first few posts - that is, nothing they write goes public until the group owner (or others they appointed) decides they seem to have applicable knowledge and are not posting spam.

In some groups everybody is moderated all the time, but in most such the initial moderation process is turned off once posters seem ok... but can be turned back on again for specific users who 'misbehave'.

[mayzie@fastmail]

Thanks Bill for all your advice!
I've been waiting till I had a few group messages in my Spam folder, and they do indeed all have spam scores ranging from 5.0 to 7.7.

Originally Posted by n5bb

Next I recommend that you select one of the groups.io messages with a spam score of 5.0 or greater so you are reading that message. Then open the Actions menu on the far right side (under the date) and select Show raw message. This will open up a new browser window which shows the raw content of that complete message. I know that this screen will look very complex, but we only need to examine a few lines which start with X-. Those X- header lines have special Fastmail-specific information which tell us the internal state of the system when the message was processed.

Here's the X-headers for one of the messages with a spam score of 7.7. I'm not exactly sure how to interpret the Bayes values. I do see the "BAYES_USED user" piece so presumably it's reflecting me clicking "not spam" a bunch of times? But I don't know what the specific numbers indicate.

X-Spam-score: 7.7
X-Spam-hits: BAYES_99 3.5, BAYES_999 1.2, FREEMAIL_FROM 0.001, HTML_MESSAGE 0.001,
MAILING_LIST_MULTI -1, ME_HAS_VSSU 0.001, ME_SC_NH -0.001,
ME_SENDERREP_DENY 4, RCVD_IN_DNSWL_NONE -0.0001, RCVD_IN_MSPIKE_H2 0.001,
RCVD_IN_PBL 0.001, SPF_HELO_NONE 0.001, SPF_PASS -0.001, LANGUAGES en,
BAYES_USED user, SA_VERSION 4.0.1
X-Spam-source: IP='209.85.221.169', Host='mail-vk1-f169.google.com', Country='US',
FromHeader='com', MailFrom='com'

[JeremyNicoll]

Originally Posted by mayzie@fastmail

Here's the X-headers for one of the messages with a spam score of 7.7. I'm not exactly sure how to interpret the Bayes values. I do see the "BAYES_USED user" piece so presumably it's reflecting me clicking "not spam" a bunch of times? But I don't know what the specific numbers indicate.

X-Spam-score: 7.7
X-Spam-hits: BAYES_99 3.5, BAYES_999 1.2, FREEMAIL_FROM 0.001, HTML_MESSAGE 0.001,
MAILING_LIST_MULTI -1, ME_HAS_VSSU 0.001, ME_SC_NH -0.001,
ME_SENDERREP_DENY 4, RCVD_IN_DNSWL_NONE -0.0001, RCVD_IN_MSPIKE_H2 0.001,
RCVD_IN_PBL 0.001, SPF_HELO_NONE 0.001, SPF_PASS -0.001, LANGUAGES en,
BAYES_USED user, SA_VERSION 4.0.1
X-Spam-source: IP='209.85.221.169', Host='mail-vk1-f169.google.com', Country='US',
FromHeader='com', MailFrom='com'

The BAYES_99 3.5 means that FM have scored this email 3.5 points because the BAYES system is more than 99% sure the mail is spam.

The BAYES_999 1.2 means that FM have scored this email 1.2 more points because the BAYES system is more than 99.9% sure the email is spam ... so these two issues alone provide 4.7 points.

The other big problem is the 4 points added by: ME_SENDERREP_DENY 4 ... which means that the sending server is thought to be truly untrustworthy (but there's no clue here why that is).


There's clearly a problem with your BAYES set-up. You said you've regularly clicked on spammy emails to tell BAYES that those were spam, but have you also OFTEN clicked on genuine emails to say that they are not-spam? If I recall, BAYES needs plenty of evidence of what's good as well as what's bad.

Also, I found - about halfway down this FM help page:
https://www.fastmail.help/hc/en-us/a...pam-protection
text saying:

Note: We recommend that you do not mark your Spam/Junk Mail folder to automatically learn as spam. This can create a false positive feedback loop. Imagine an email is incorrectly classified as spam, put in your Spam/Junk Mail folder, and then learned as spam. That means future emails that aren't spam are now more likely to be incorrectly marked as spam, sent to your Spam/Junk Mail folder, and learned as spam. Only mark folders to learn as spam if they're folders you manually move email to.

You'd need to read the whole page to see what this is about, but if you've made that mistake then this would not help...

I don't know if FM offer a way for you to delete the BAYES database so that its learning of what is/isn't spam can start all over again, but that might be a good idea.



ALSO can you post corresponding headers from a groups.io mail that's been flagged as spam?

[n5bb]

I agree with all of the comments by JeremyNicoll.

• That was obviously a message sent from Gmail. The X-Spam-source header is similar to typical Gmail good messages I receive. Please look at the contents of that message. Does it appear to be a good message, or is it a spam or phishing message? Do you know the sender? If so, you might ask them (if possible in a different way, such as over the telephone or using a different non-Gmail email address) if they actually sent that message. If it is a good non-spam message, then something is very wrong due to those three headers with unusual values (BAYES_99, BAYES_999, and ME_SENDERREP_DENY).
• Also look at this Fastmail help topic:
  Improving spam protection
  Scroll down to Your personal spam database and tell us how many spam and non-spam messages have been reported.
• Years ago Fastmail had a button for users to reset their personal Bayes spam filter, but this feature is no longer available.
• If you have a similar issue with groups.io messages, I think you will need to file a Fastmail support request and get their staff to see what?s going wrong with your spam filtering.

Bill

[mayzie@fastmail]

Originally Posted by JeremyNicoll

There's clearly a problem with your BAYES set-up. You said you've regularly clicked on spammy emails to tell BAYES that those were spam, but have you also OFTEN clicked on genuine emails to say that they are not-spam? If I recall, BAYES needs plenty of evidence of what's good as well as what's bad.

No - I didn't say I regularly click on spammy emails to tell BAYES they're spam (I rarely get spam in my Inbox). What I've been doing regularly is clicking on "Not spam" for good emails (usually googlegroups or groups.io) that go into my Spam folder. So I think I have been giving BAYES plenty of evidence of what's good.

I'd really love to know why this is messing up my BAYES scores!

One thing I haven't mentioned is that most of mail is forwarded from other accounts to my Fastmail account. But I have added the SMTP addresses for each of those accounts in the "Forwarding Hosts" field under Spam Protection (advanced settings), as Fastmail recommends.

Originally Posted by JeremyNicoll

Also, I found - about halfway down this FM help page:
https://www.fastmail.help/hc/en-us/a...pam-protection
text saying:

Note: We recommend that you do not mark your Spam/Junk Mail folder to automatically learn as spam. This can create a false positive feedback loop. Imagine an email is incorrectly classified as spam, put in your Spam/Junk Mail folder, and then learned as spam. That means future emails that aren't spam are now more likely to be incorrectly marked as spam, sent to your Spam/Junk Mail folder, and learned as spam. Only mark folders to learn as spam if they're folders you manually move email to.

Again, this is about trying to better identify Spam emails, which is the reverse of my problem (and I don't do this).

That Help page says:

Every so often, it's a good idea to check your Spam folder to see if any legitimate messages have been accidentally flagged as spam. If it has, select the message and click the Not spam button to mark it as not a spam message.

... which is exactly what I do. Also, I'm generally working directly in Fastmail's mobile app when I do this.

I've also set up a Rule to try and steer incoming group emails directly to my Inbox. (Since these emails show individual sender addresses in the From line, and the group address in the To line, I use the To line as the condition.) I've tried leaving this rule enabled and disabling it, but it doesn't seem to make any difference either way.

Originally Posted by JeremyNicoll

I don't know if FM offer a way for you to delete the BAYES database so that its learning of what is/isn't spam can start all over again, but that might be a good idea.

Seems like this might be the best approach if I can figure out how to do it. But I'd also like to know what I did wrong so I don't repeat it!

Originally Posted by JeremyNicoll

ALSO can you post corresponding headers from a groups.io mail that's been flagged as spam?

Unfortunately I don't have any groups.io mail in my Spam folder right now - I've always clicked "Not spam" when they come in, which moves them to my Inbox. Next time I'll try checking the headers first.