SpamAssassin 3.0 - better spam protection

Jeremy Howard · 26 Sep 2004, 02:15 PM

A quick note to let you know we've upgraded to SpamAssassin 3.0. More information shortly - but in summary, you should find spam filtering much improved! :-) It's working for both POP links, and incoming mail.

pingme97 · 26 Sep 2004, 03:34 PM

Quote:

Originally posted by Jeremy Howard
A quick note to let you know we've upgraded to SpamAssassin 3.0. More information shortly - but in summary, you should find spam filtering much improved! :-) It's working for both POP links, and incoming mail.

Thanks Jeremy for the upgrade. you people are simply great

any news on other upgrades

fmnewbee · 26 Sep 2004, 05:31 PM

Thanks for updating us on this.

I would like to know how the programmers of the new version has dealt with the false positives of the older versions?

1. In Subject line I wrote:

Test of Swedish chars åäö
(I mean why would that be seen as a spam. )

2. To stop dictionary attacks of my name I throw in random numbers i nthe name like.

username John.Doe was changed to j5oh3n.7d1oe
the 5371 being the code for the entry at my old place of living. Spam Assassine saw it as a spammer tactic. When it in reality was the opposite. Trying to stop spam.

3. My Subject line started with Hi. (ok I know now that to be a typical spammers way of disguising themselves to know me but many others who are newbes don't know thus unknowingly creating a false positive)

Hope they have addressed such peculiar misguided attampts.

Trew

mcowger · 27 Sep 2004, 01:30 AM

Quote:

Originally posted by fmnewbee
Thanks for updating us on this.

I would like to know how the programmers of the new version has dealt with the false positives of the older versions?

1. In Subject line I wrote:

Test of Swedish chars åäö
(I mean why would that be seen as a spam. )

2. To stop dictionary attacks of my name I throw in random numbers i nthe name like.

username John.Doe was changed to j5oh3n.7d1oe
the 5371 being the code for the entry at my old place of living. Spam Assassine saw it as a spammer tactic. When it in reality was the opposite. Trying to stop spam.

3. My Subject line started with Hi. (ok I know now that to be a typical spammers way of disguising themselves to know me but many others who are newbes don't know thus unknowingly creating a false positive)

Hope they have addressed such peculiar misguided attampts.

Trew

You can find out exactly why a specific message was determined to be spam by checking the headers and seeing which tests were triggered.

fmnewbee · 27 Sep 2004, 03:50 AM

But a structured noob would be able t ouse the info while a non-structured noob like me see al lthe words but are not good at making a whole out of all the details.

X-Spam-Report:
* 2.0 FROM_HAS_MIXED_NUMS3 From: contains numbers mixed in with letters * 0.3 FROM_HAS_MIXED_NUMS From: contains numbers mixed in with letters
X-Spam-Status:
No, hits=2.3 required=5.0 tests=FROM_HAS_MIXED_NUMS, FROM_HAS_MIXED_NUMS3

here is another also from Spam Assassin

X-Spam-Report:
Spam detection software, running on the system "postaccess-01", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or block similar future email. If you have any questions, see the administrator of that system for details.

Content preview: test åäö ok Trew Bumblebee Sweden [...] Content analysis details: (0.0 points, 4.0 required) pts rule name description

I wrote test of our swedish accented letters or chars åäö ok and that triggered that version of SpamAssassin to think of me as a typical spammer.

I find it to be overkill. why have such crazy rules.

so I am very interested in the approach of Crunchmail? and them using another tech that is less pronefull I hope?

Rob_au referred to it.

http://www.emaildiscussions.com/...ighlight=Trust

"Trust metrics in email delivery processes"
Trew

PS
postaccess is old Portland the free host that also has free email service. But the first example was Swissinfo.org email service, both of these use Spam Assassine.

Jeremy Howard · 27 Sep 2004, 07:47 AM

Trew, neither of the messages you show above are false positives. SpamAssassin is designed to filter with a threshold of 5.0 or higher. Neither of these messages have such a high score.

The idea of SpamAssassin is to have lots of rules which are somewhat associated with spam (or non-spam) and combine them using a score to determine "spaminess". You should not rely on any individual rule, but should use the total score.

The scores were trained using a statistical algorithm (a perceptron) to find the scores that best sorted spam from non-spam for a large group of people. Not every score will be perfect for your particular emails, but overall the combined score should be useful.

Aristarchus · 30 Sep 2004, 03:50 AM

I'm not sure how everyone else is doing, but my spam filter has gotten a lot worse since the upgrade. Most notably there have been a lot more false negatives (spam that gets through the filter) than before.

Brian11480 · 30 Sep 2004, 04:41 AM

I didn't realize SpamAssassin got upgraded, Maybe this is the reason for my ten-fold increase in SPAM getting to my inbox.

i would be in favor of a downgrade.

marcmartin · 30 Sep 2004, 04:54 AM

Yes, I thought that the spam filter had just stopped working a few days ago, but I guess the "upgrade" explains it. I guess a lot of spams are now scoring a lot lower than before, because my inbox has a lot more spams than before, and they are getting scored in the ~2 range.

Brian11480 · 30 Sep 2004, 07:26 AM

With all this extra spam coming in I really can use that extra storage space cause all this Spam is filling up my inbox.

Tomorrow is the last day of Sept. It can't come soon enough.

audelair · 1 Oct 2004, 01:35 AM

yea, the spam is really taking over my inbox. i'd really like to see the old spamassassin again

SusanUKF · 1 Oct 2004, 01:38 AM

Thanks for the upgrade Jeremy, it is nice to see things improving all the time!!

Susan

Brian11480 · 1 Oct 2004, 02:32 AM

Quote:

Originally posted by SusanUKF
Thanks for the upgrade Jeremy, it is nice to see things improving all the time!!

Susan

Were you being sarcastic in that statement?

I seriously have had a ten-fold increase in Spam getting into my inbox since the upgrade.

mcowger · 1 Oct 2004, 02:34 AM

Quote:

Originally posted by Brian11480
Were you being sarcastic in that statement?

I seriously have had a ten-fold increase in Spam getting into my inbox since the upgrade.

I don't think she was. I'm not. I've seen less spam, so its not a system wide problem.

audelair · 1 Oct 2004, 02:34 AM

Quote:

Originally posted by Brian11480
Were you being sarcastic in that statement?

I seriously have had a ten-fold increase in Spam getting into my inbox since the upgrade.

she jumps up for joy when the server comes back after a one hour outage. don't pay attention to her.

I agree, this new Spamassassin upgrade has been very troublesome. I hesitate to change my spam settings because of this though.

26 Sep 2004, 02:15 PM	#1
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	SpamAssassin 3.0 - better spam protection A quick note to let you know we've upgraded to SpamAssassin 3.0. More information shortly - but in summary, you should find spam filtering much improved! :-) It's working for both POP links, and incoming mail.

26 Sep 2004, 05:31 PM	#3
fmnewbee Master of the @ Join Date: Aug 2004 Location: Sweden Posts: 1,135	How do they get less amount of false posistive? Thanks for updating us on this. I would like to know how the programmers of the new version has dealt with the false positives of the older versions? 1. In Subject line I wrote: Test of Swedish chars åäö (I mean why would that be seen as a spam. ) 2. To stop dictionary attacks of my name I throw in random numbers i nthe name like. username John.Doe was changed to j5oh3n.7d1oe the 5371 being the code for the entry at my old place of living. Spam Assassine saw it as a spammer tactic. When it in reality was the opposite. Trying to stop spam. 3. My Subject line started with Hi. (ok I know now that to be a typical spammers way of disguising themselves to know me but many others who are newbes don't know thus unknowingly creating a false positive) Hope they have addressed such peculiar misguided attampts. Trew

27 Sep 2004, 03:50 AM	#5
fmnewbee Master of the @ Join Date: Aug 2004 Location: Sweden Posts: 1,135	Yes I know that is why I did know :) But a structured noob would be able t ouse the info while a non-structured noob like me see al lthe words but are not good at making a whole out of all the details. X-Spam-Report: * 2.0 FROM_HAS_MIXED_NUMS3 From: contains numbers mixed in with letters * 0.3 FROM_HAS_MIXED_NUMS From: contains numbers mixed in with letters X-Spam-Status: No, hits=2.3 required=5.0 tests=FROM_HAS_MIXED_NUMS, FROM_HAS_MIXED_NUMS3 here is another also from Spam Assassin X-Spam-Report: Spam detection software, running on the system "postaccess-01", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or block similar future email. If you have any questions, see the administrator of that system for details. Content preview: test åäö ok Trew Bumblebee Sweden [...] Content analysis details: (0.0 points, 4.0 required) pts rule name description I wrote test of our swedish accented letters or chars åäö ok and that triggered that version of SpamAssassin to think of me as a typical spammer. I find it to be overkill. why have such crazy rules. so I am very interested in the approach of Crunchmail? and them using another tech that is less pronefull I hope? Rob_au referred to it. http://www.emaildiscussions.com/...ighlight=Trust "Trust metrics in email delivery processes" Trew PS postaccess is old Portland the free host that also has free email service. But the first example was Swissinfo.org email service, both of these use Spam Assassine. Last edited by fmnewbee : 27 Sep 2004 at 04:10 AM.

27 Sep 2004, 07:47 AM	#6
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	Trew, neither of the messages you show above are false positives. SpamAssassin is designed to filter with a threshold of 5.0 or higher. Neither of these messages have such a high score. The idea of SpamAssassin is to have lots of rules which are somewhat associated with spam (or non-spam) and combine them using a score to determine "spaminess". You should not rely on any individual rule, but should use the total score. The scores were trained using a statistical algorithm (a perceptron) to find the scores that best sorted spam from non-spam for a large group of people. Not every score will be perfect for your particular emails, but overall the combined score should be useful.

30 Sep 2004, 03:50 AM	#7
Aristarchus Member Join Date: Apr 2003 Posts: 43	I'm not sure how everyone else is doing, but my spam filter has gotten a lot worse since the upgrade. Most notably there have been a lot more false negatives (spam that gets through the filter) than before.

30 Sep 2004, 04:41 AM	#8
Brian11480 Essential Contributor Join Date: Oct 2002 Location: Albany, NY, USA Posts: 251	I didn't realize SpamAssassin got upgraded, Maybe this is the reason for my ten-fold increase in SPAM getting to my inbox. i would be in favor of a downgrade.

30 Sep 2004, 04:54 AM	#9
marcmartin Junior Member Join Date: Sep 2004 Posts: 10	Yes, I thought that the spam filter had just stopped working a few days ago, but I guess the "upgrade" explains it. I guess a lot of spams are now scoring a lot lower than before, because my inbox has a lot more spams than before, and they are getting scored in the ~2 range.

30 Sep 2004, 07:26 AM	#10
Brian11480 Essential Contributor Join Date: Oct 2002 Location: Albany, NY, USA Posts: 251	With all this extra spam coming in I really can use that extra storage space cause all this Spam is filling up my inbox. Tomorrow is the last day of Sept. It can't come soon enough.

1 Oct 2004, 01:35 AM	#11
audelair Essential Contributor Join Date: Feb 2004 Posts: 271	yea, the spam is really taking over my inbox. i'd really like to see the old spamassassin again

1 Oct 2004, 01:38 AM	#12
SusanUKF Intergalactic Postmaster Join Date: Oct 2003 Location: Canada Posts: 5,428	Thanks for the upgrade Jeremy, it is nice to see things improving all the time!! Susan