Cutstom spam settings and adding contact to address book

[sarf]

Despite adding the contacts to my address book I am getting legit mail sent to the spam folder . Marking these as "not spam" and supposedly training the spam tool makes little or no difference .
One option seems to be to raise the the threshold for mail going into the spam folder to avoid those false positives but then there's the danger of false negatives slipping in as legit email.
In terms of getting it right re what is or isn't spam fastmail seems to have got worse in the last couple of years.
I would be interested to know what custom settings people use to minimise false positives and false negatives. Mine currently is 7.5 discard completely and 5.5 file into spam folder .

[n5bb]

There is currently a problem with a few false positives (good messages accidentally filed into your spam folder) due to a couple of problems with the new DMARC processing. I believe that one of the bugs has been fixed, and the other should be fixed within the next day.

So please wait another day or two and then see what is happening. If you still see problems, look for consistent problems with the content of the following headers (use More > Show Raw Message to see these headers):

X-Spam-known-sender:
X-Spam-score:
X-Spam-hits:

In particular, in a couple of days if you still see in the X-Spam-known-sender some text in quotes (such as "Email failed DMARC policy for domain" or "From == To and no DKIM or SPF for from domain, likely forged") and you are positive that the message is legitimate after looking at the other headers (such as X-Spam-source, which shows the originating details for the message), please post here. The new system may cause some messages forwarded by other systems to be classified as spam if the message body is modified by the forwarding system, due to DKIM authentication failure if DMARC is active for the sending domain. Sorry for so much technical jargon!

Bill

[sarf]

Most posts to a mental health forum still going into spam folder despite moving posts to a folder that is trained as "non spam" .

Here are headers from some of the posts.

X-Spam-known-sender: no, "From == To and no DKIM or SPF for from domain, likely forged"
Subject: {SPAM 05.7} [Remix talk] Mental health services must get the funding
they need
X-Spam: spam
X-Spam-score: 5.7
X-Spam-hits: BAYES_99 3.5, BAYES_999 1.2, ME_FROM_EQ_TO 0.01,
RCVD_IN_DNSWL_NONE -0.0001, RCVD_IN_UNSUBSCOREBL 1, SPF_PASS -0.001,
LANGUAGES en, BAYES_USED user, SA_VERSION 3.3.2
X-Spam-source: IP='192.185.50.73', Host='gateway24.websitewelcome.com', Country='US',
FromHeader='com', MailFrom='com


X-Spam-known-sender: no, "From == To and no DKIM or SPF for from domain, likely forged"
Subject: {SPAM 05.7} [Remix talk] Mental health services must get the funding
they need
X-Spam: spam
X-Spam-score: 5.7
X-Spam-hits: BAYES_99 3.5, BAYES_999 1.2, ME_FROM_EQ_TO 0.01,
RCVD_IN_DNSWL_NONE -0.0001, RCVD_IN_UNSUBSCOREBL 1, SPF_PASS -0.001,
LANGUAGES en, BAYES_USED user, SA_VERSION 3.3.2
X-Spam-source: IP='192.185.185.36', Host='gateway36.websitewelcome.com', Country='US',
FromHeader='com', MailFrom='com'

X-Spam-known-sender: no, "From == To and no DKIM or SPF for from domain, likely forged"
Subject: {SPAM 05.7} [Remix talk] Mental health workers don't listen
X-Spam: spam
X-Spam-score: 5.7
X-Spam-hits: BAYES_99 3.5, BAYES_999 1.2, ME_FROM_EQ_TO 0.01,
RCVD_IN_DNSWL_NONE -0.0001, RCVD_IN_UNSUBSCOREBL 1, SPF_PASS -0.001,
LANGUAGES en, BAYES_USED user, SA_VERSION 3.3.2
X-Spam-source: IP='192.185.51.31', Host='gateway24.websitewelcome.com', Country='US',
FromHeader='com', MailFrom='com'


The ' training' seems to be doing very little with only about 5% of the posts going into the correct folder.

[n5bb]

As I said:

Originally Posted by n5bb

The new system may cause some messages forwarded by other systems to be classified as spam if the message body is modified by the forwarding system, due to DKIM authentication failure if DMARC is active for the sending domain

Note "From == To and no DKIM or SPF for from domain, likely forged". Address book whitelisting (placing good senders in your address book) is now disabled if the message appears to be spoofed (not actually sent by the From sender). I can't see the From domain in the headers you posted, so I can't check their DMARC policy. It appears from the limited information you posted that the sender's domain has a DMARC policy, and forwarding services or message forum forwarding in many cases may cause those messages to be rejected as not meeting the sending domain policy. This would also affect messages received at Gmail and many other email services.
https://dmarc.org/2015/10/global-mai...protect-users/

But something else is wrong -- in the headers you did post the use Bayes system is classifying those messages as 99.9% likely to be spam (BAYES_99 and BAYES_999). This is rather unusual, and my guess is that either you have accidentally been classifying similar messages as spam or that your user Bayes database is corrupted. I suggest the following:

• Check the folder where you are filing these messages and be sure it is set up as: Scan this folder daily and learn any new messages as not spam
• Only new messages arriving in a folder set up in this fashion will affect the user Bayes score. If you have a large number of such messages, set up a temporary folder (which does not learn spam), then move the messages similar to this one to the temporary folder, then move those messages back to the folder which is verified to learn then as not spam. I think this will force them to be not spam after the next daily scan, so be sure to leave them in the "learning as not spam" folder for at least 24 hours.

Another problem is the "RCVD_IN_UNSUBSCOREBL" SpamAssassin score. This indicates that the forum probably isn't responding correctly to unsubscribe requests:
http://www.dnsbl.info/dnsbl-details....unsubscore.com

Bill

[sarf]

The from domain is hostgator.com . I don't have a large number of such emails as after transferring them to the bonkersbob folder (set up to scan new messages daily as non spam) I delete them the next day.

Just to clarify I should move such posts to a learning not spam folder and then to bonkers bob folder after 24v hours or more with both set up to learn any messages as non spam?

[n5bb]

Well, my DMARC guess was incorrect, since "hostgator.com" publishes no DMARC policy. They do publish a SPF record, but obviously the SPF test is failing to align with the From domain (probably due to the post being forwarded).

I'm not sure how you somehow trained your Bayes folder to dislike these messages so badly. I'm worried about your comments about moving messages which were classified as spam. The only way you should deal with false positives (good messages appearing in your Spam/Junk Mail folder) is to use the Not Spam button at the top on those messages. If you delete the good messages rather than marking them as Not Spam and your Spam/Junk Mail folder is set to learn as spam, you will be training your Bayes filter that those good messages are spam.

Is your Trash folder set up to learn messages as spam? If so, that would explain everything. The last action taken on a message (marking it manually or automatically as spam or not spam) is what affects your Bayes spam filter. I think it's very important that you again check the advanced folder properties and verify which folders are learning as spam.

My suggestion about relearning messages is only useful if have a lot of such messages stored in a folder.

Bill

[sarf]

Bill, my trash folder is not learning as spam . The folders learning as spam are a folder named "learn spam " where I am moving false negatives to.
https://www.fastmail.com/help/receiv...ac#spamfolders

I am mostly using thunderbird rather than the fastmail web site .

[n5bb]

If you are very careful that should work, as long as you are also moving false positives (the messages you posted) to a folder which learns as not spam. But you need to leave messages in a folder over 24 hours to insure it learns.

It appears to me that something must have gone wrong to get such a poor Bayes score for those messages. It's harder to troubleshoot this if you are using an email client. You might want to log into the web interface to mark the false positives as Not Spam to be sure they are learned correctly.

Bill

[sarf]

Originally Posted by n5bb

If you are very careful that should work, as long as you are also moving false positives (the messages you posted) to a folder which learns as not spam. But you need to leave messages in a folder over 24 hours to insure it learns.

I also have a 'learn as not spam' folder into which I am moving false positives. Both are set to auto purge after 3 days. However this was not working. Perhaps because the folders were set to default setting for the account in thunderbird rather than delete messages older than 3 days

[sarf]

Just had 3 notifications from bonkers bob two went into spam and one didn't. The two that went into spam had a score of 5.7 - X-Spam-known-sender: no, "From == To and no DKIM or SPF for from domain, likely forged"
Subject: {SPAM 05.7} [Remix talk] FDA warns about rare but serious skin reactions
with mental health drug olanzapi
X-Spam: spam
X-Spam-score: 5.7
X-Spam-hits: BAYES_99 3.5, BAYES_999 1.2, ME_FROM_EQ_TO 0.01,
RCVD_IN_DNSWL_NONE -0.0001, RCVD_IN_UNSUBSCOREBL 1, SPF_PASS -0.001,
LANGUAGES en, BAYES_USED user, SA_VERSION 3.3.2

while the one that didn't had a spam score of 4.7 - X-Sieve: CMU Sieve 2.4
X-Spam-known-sender: no, "From == To and no DKIM or SPF for from domain, likely forged"
X-Spam-score: 4.7
X-Spam-hits: BAYES_99 3.5, BAYES_999 1.2, ME_FROM_EQ_TO 0.01,
RCVD_IN_DNSWL_NONE -0.0001, SPF_PASS -0.001, LANGUAGES en,
BAYES_USED user, SA_VERSION 3.3.2