SpamAssassin Y2010 Bug

jgoldberg · 2 Jan 2010, 07:01 AM

There is is (very recently) known bug in SpamAssassin rules and this bug is affecting at least one FM user (me).

Here is an excerpt from one of my headers.

Code:

X-Spam-Hits: BAYES_00 -2.599, FH_DATE_PAST_20XX 3.188, RCVD_IN_DNSWL_MED -4, SPF_PASS -0.001, BAYES_USED user

The offending part is the

Code:

FH_DATE_PAST_20XX 3.188

part.

Basically there is a rule in the standard SpamAssassin distribution that considers Date headers with a year greater than 2009 as "far in the future".

I've posted details about this at

http://jpgoldberg.blogspot.com/2010/...y2010-bug.html

Reading discussion on the spam assassin mailing list, it may take time before this rule is corrected through sa-update, but in the meantime 3 points can lead to a lot of non-spam being misidentified. I recommend disabling this rule as soon as possible.

n5bb · 2 Jan 2010, 07:22 AM

Good catch, Jeffrey. I just sent Rob and Bron at Fastmail a message based on your post.

I wonder if we have gotten the last snow in North Texas this season? HNY!

Bill

EdinwolfPA · 2 Jan 2010, 07:36 AM

Hello,

There appears to be a similar thread on the same topic:

http://www.emaildiscussions.com/show...765#post494765

Is this the same issue or different??

IF it is the same issue there might be some useful information in the other post from other forum members

Just curious

David

jgoldberg · 2 Jan 2010, 07:56 AM

Quote:

Originally Posted by EdinwolfPA

There appears to be a similar thread on the same topic:

http://www.emaildiscussions.com/show...765#post494765

Is this the same issue or different??

It's the same. Sorry, I had looked around before posting, but hadn't seen that.

EdinwolfPA · 2 Jan 2010, 08:09 AM

Quote:

Originally Posted by jgoldberg

It's the same. Sorry, I had looked around before posting, but hadn't seen that.

Hello,

It was in a forum that no one really uses to any great extent-

Just thought I would pass the information along to you-that there appears to be a patch and/or fix already according to another forum member Janusz and to inform the FM team in the event they are looking at this issue on the forum.

David

ao1 · 2 Jan 2010, 08:10 AM

A call to the FM team to adjust FH_DATE_PAST_20XX.

brong · 2 Jan 2010, 08:28 AM

Hi,

Spamassassin have released an update that fixes it, and an apology for their messing up on sending the fix out!

https://issues.apache.org/SpamAssass...ug.cgi?id=6269

I've updated our servers and restarted the spam scanning service:

OLD:

Code:

529:##{ FH_DATE_PAST_20XX
530:header   FH_DATE_PAST_20XX  Date =~ /20[1-9][0-9]/ [if-unset: 2006]
531:describe FH_DATE_PAST_20XX  The date is grossly in the future.
532:##} FH_DATE_PAST_20XX

NEW:

Code:

529:##{ FH_DATE_PAST_20XX
530:header   FH_DATE_PAST_20XX	Date =~ /20[2-9][0-9]/ [if-unset: 2006]
531:describe FH_DATE_PAST_20XX	The date is grossly in the future.
532:##} FH_DATE_PAST_20XX

It will now only match dates past 2020, which gives us breathing room for the SA team to come up with a more flexible solution...

n5bb · 2 Jan 2010, 08:37 AM

Thanks, Bron! It's fixed on my incoming email now. Sorry to bug you on a holiday -- oops, it's a holiday for me, but it's just a weekend day for you!

Bill

brong · 2 Jan 2010, 08:43 AM

Yeah - regular weekend

I don't mind fixing major issues any time - and at least it's during the day! We lost two hard disks in a RAID1 on Christmas Day, that was a bit more special... thankfully after the kids were in bed, but it took 4 hours to get everything back up and running.

Yay for replication and almost seamless cutover, so nobody except for our sysadmins was aware of it!

ChinaLamb · 2 Jan 2010, 02:09 PM

Quote:

Originally Posted by brong

Yeah - regular weekend

I don't mind fixing major issues any time - and at least it's during the day! We lost two hard disks in a RAID1 on Christmas Day, that was a bit more special... thankfully after the kids were in bed, but it took 4 hours to get everything back up and running.

Yay for replication and almost seamless cutover, so nobody except for our sysadmins was aware of it!

Thanks for sacrificing your holiday to keep us online! Love the reliability - never knew anything was wrong.

digp · 2 Jan 2010, 07:36 PM

Quote:

Originally Posted by brong

... We lost two hard disks in a RAID1 on Christmas Day, that was a bit more special...

What went wrong? Why did that happen?

ChinaLamb · 2 Jan 2010, 08:13 PM

Quote:

Originally Posted by digp

What went wrong? Why did that happen?

Actually quite normal for hard disks to fail. That's where RAID comes in. Because the disks are under almost constant use, they eventually fail. RAID keeps information replicated across an array of disks. You can take one disk out and replace it with a new one and the RAID controller will again replicate the data as necessary with the new disk. Under normal conditions you can replace a disk and no one will ever know a disk went bad. Sometimes, disks are near failure and will fail when another disk fails...

This is why entire servers are also replicated. Occasionally with multiple disks failing at once, the failure is catastrophic. If another server is set up as a replicated server, you can just transfer all work to the other server while you fix the first one.

Anyone that has worked with servers long enough has seen something like a multiple disk failure in a raid array... Don't need anything special to happen, other than for the disks to have worn out of their useful life...

I think this incident shows that Fastmail's replication setup is working very well.

2 Jan 2010, 07:01 AM	#1
jgoldberg Member Join Date: Jul 2003 Location: Plano, Texas Posts: 88	SpamAssassin Y2010 Bug There is is (very recently) known bug in SpamAssassin rules and this bug is affecting at least one FM user (me). Here is an excerpt from one of my headers. Code: X-Spam-Hits: BAYES_00 -2.599, FH_DATE_PAST_20XX 3.188, RCVD_IN_DNSWL_MED -4, SPF_PASS -0.001, BAYES_USED user The offending part is the Code: FH_DATE_PAST_20XX 3.188 part. Basically there is a rule in the standard SpamAssassin distribution that considers Date headers with a year greater than 2009 as "far in the future". I've posted details about this at http://jpgoldberg.blogspot.com/2010/...y2010-bug.html Reading discussion on the spam assassin mailing list, it may take time before this rule is corrected through sa-update, but in the meantime 3 points can lead to a lot of non-spam being misidentified. I recommend disabling this rule as soon as possible.

2 Jan 2010, 07:36 AM	#3
EdinwolfPA Cornerstone of the Community Join Date: Oct 2009 Location: PA Posts: 683	Question for you- Is this the same issue?? Hello, There appears to be a similar thread on the same topic: http://www.emaildiscussions.com/show...765#post494765 Is this the same issue or different?? IF it is the same issue there might be some useful information in the other post from other forum members Just curious David Last edited by EdinwolfPA : 2 Jan 2010 at 07:56 AM.

2 Jan 2010, 08:28 AM	#7
brong The "e" in e-mail Join Date: Jul 2004 Location: Melbourne, Australia Posts: 2,696 Representative of: Fastmail.fm	Hi, Spamassassin have released an update that fixes it, and an apology for their messing up on sending the fix out! https://issues.apache.org/SpamAssass...ug.cgi?id=6269 I've updated our servers and restarted the spam scanning service: OLD: Code: 529:##{ FH_DATE_PAST_20XX 530:header FH_DATE_PAST_20XX Date =~ /20[1-9][0-9]/ [if-unset: 2006] 531:describe FH_DATE_PAST_20XX The date is grossly in the future. 532:##} FH_DATE_PAST_20XX NEW: Code: 529:##{ FH_DATE_PAST_20XX 530:header FH_DATE_PAST_20XX Date =~ /20[2-9][0-9]/ [if-unset: 2006] 531:describe FH_DATE_PAST_20XX The date is grossly in the future. 532:##} FH_DATE_PAST_20XX It will now only match dates past 2020, which gives us breathing room for the SA team to come up with a more flexible solution...

2 Jan 2010, 07:22 AM	#2
n5bb Intergalactic Postmaster Join Date: May 2004 Location: Irving, Texas Posts: 8,929	Good catch, Jeffrey. I just sent Rob and Bron at Fastmail a message based on your post. I wonder if we have gotten the last snow in North Texas this season? HNY! Bill

2 Jan 2010, 08:10 AM	#6
ao1 Essential Contributor Join Date: Oct 2003 Posts: 327	A call to the FM team to adjust FH_DATE_PAST_20XX.

2 Jan 2010, 08:37 AM	#8
n5bb Intergalactic Postmaster Join Date: May 2004 Location: Irving, Texas Posts: 8,929	Thanks, Bron! It's fixed on my incoming email now. Sorry to bug you on a holiday -- oops, it's a holiday for me, but it's just a weekend day for you! Bill

2 Jan 2010, 08:43 AM	#9
brong The "e" in e-mail Join Date: Jul 2004 Location: Melbourne, Australia Posts: 2,696 Representative of: Fastmail.fm	Yeah - regular weekend I don't mind fixing major issues any time - and at least it's during the day! We lost two hard disks in a RAID1 on Christmas Day, that was a bit more special... thankfully after the kids were in bed, but it took 4 hours to get everything back up and running. Yay for replication and almost seamless cutover, so nobody except for our sysadmins was aware of it!