Trainable spam filter ready for beta testing

[trond]

We're now ready to start beta testing of the trainable spamfilter. We're using a system called DSpam (http://www.nuclearelephant.com/projects/dspam/)

If anybody wants to test this new filter, send me a personal message here at the forum, and tell me your username.
After I've set up the system to filter your mail through dspam, you'll get an extra checkbox in the filter page that says "use trainable spamfilter". You'll have to check this, and then "save settings" to enable the trainable filter.

Beware that the normal behaviour of a trainable filter is to first be too lenient, and let pretty much everything slip through. After a while it'll start catching spam, and then it's likely to be too strict, and start catching legitimate mail - particularly newsletters and similar. So you need to check the spam folder regularly in the beginning, to make sure that there are no legitimate mails there. After a few hundred mails it'll start to get really good at recognizing the mails you don't want, and you can start ignoring your spamfolder.

Another thing to remember is that the filter trains itself, so if it misclassifies a mail, and you don't correct it, it's likely to repeat the mistake. In particular, if it starts to miss spams, it'll get increasingly worse if you just delete the spam instead of using the "report spam" button.

[BLuRReD]

are u willing to add a buttom called this is not spam as well ?

[trond]

There is one already

In the spam folder, the button says "not spam", everywhere else, it says "report spam"

[BLuRReD]

okay , another question , does this mean u creat a new folder called spam for users , or at least for beta users now or what ?

[trond]

No.

You decide for yourself what folder you want spam to be directed into. Just like before, really.

[BLuRReD]

but will i be able to report any email as a spam located in any other folder , right ?

[trond]

right.

The folder you direct spam to, is considered to be your spam folder. All other folders are not.

When training, you can train every mail already in the spam folder as "not spam". All mails that are not in the spam folder can be trained as spam.

[carverrn]

Originally posted by trond
There is one already

In the spam folder, the button says "not spam", everywhere else, it says "report spam"

Minor issue ... when viewing a message my "Report Spam" link splits like this:

Inbox Compose Reply Reply to all Forward Delete Read/Unread Report
Spam Retrieve POP

Could you either make "Report Spam" non-breaking or just bite the bullet and make this a double line all the time:

Message View
Inbox Compose Reply Reply to all Forward Delete Read/Unread
Report Spam Retrieve POP

and

spam Compose Reply Reply to all Forward Delete Read/Unread
Not Spam Retrieve POP


Folder Listing
Refresh Compose Reply Reply to all Forward Delete Read/Unread
Report Spam Retrieve POP

and

Refresh Compose Reply Reply to all Forward Delete Read/Unread
Not Spam Retrieve POP

Thanks,
Rich

[carverrn]

Trond,

I turned on the DSPAM option this morning. Saw that I was getting messages with the X-DSPAM headers in them as expected.

Just a little while ago I received a new message that does not have the X-DSPAM headers. It does hava the SpamAssassin X-Spam headers.

Any idea why it didn't get the X-DSPAM headers?

Regards,
Rich

[LrdVader]

So how does this fit in with SpamAssassin? What happens when one filter thinks a message is spam and the other doesn't?

[trond]

Spamassassin has been set up to assist in training the trainable filter.

If the trainable thinks it's innocent, and Spamassassin thinks it's spam, the message is automatically trained as spam.
If the trainable thinks it's spam, but the sender is in the whitelist, it's automatically trained as not spam.

That means that, at least to begin with, it's best to not use the "reject if possible" option. When you think the filter has matured, you can turn on rejection again, so that you'll have less spam to wade through, looking for false positives.

[LrdVader]

Originally posted by trond
If the trainable thinks it's innocent, and Spamassassin thinks it's spam, the message is automatically trained as spam.

I'm not so sure that's a good idea. The whole point of a trainable filter is that it can ultimately be smarter than SA, and be correct when it is wrong, right? It seems that having SA train it kind of defeats this purpose. And it means that using the "Not Spam" button to train the filter to avoid false positives won't accomplish much, since the filter will be re-trained to see them as spam the next time SA sees them.

Am I missing something?

[carverrn]

Originally posted by LrdVader
I'm not so sure that's a good idea. The whole point of a trainable filter is that it can ultimately be smarter than SA, and be correct when it is wrong, right?

I think they are just using Spam Assassin to "jump start" the training process.

Rich

[carverrn]

14 more messages have arrived that do not have the X-DSPAM headers in them.

Rich

[Geir]

SA has proven too lenient (e.g. excessive false negatives) and unreliable at lower thresholds than 4-5 (5 is currently default), while false positives have been very rare. Thus, Dspam is set up to catch spam that bypasses SA, in such a way that SA assists the user in training Dspam (to accelerate learning).

In the event of a false SA-positive, a "Not spam" action will SA-whitelist the sender to prevent subsequent misclassification.

If you'd like to see how great it works, PM Trond or me.

- Geir