How To Improve SPAM Filtering using Runbox Filters

RFC = Request For Comments
list
These documents set the current and future standards (define the protocols, etc).

RFC2821 (SMTP) states (Section 2.4):

The local-part of a mailbox MUST BE treated as case sensitive.

[Gnome]

Ahh cool, I'll read through that. The whitelist section should allow us to add case sensitive addresses then, it currently only alllows entries of all lower case.

[trond]

Originally posted by Daniel S
RFC2821 (SMTP) states (Section 2.4):

(that local-parts are case sensitive)

Yes, that's true, but as the next passage says:

Therefore, SMTP implementations MUST take care to preserve the case of mailbox local-parts. Mailbox domains are not case sensitive. In particular, for some hosts the user "smith" is different from the user "Smith". However, exploiting the case sensitivity of mailbox local-parts impedes interoperability and is discouraged.

So this just says that the mailserver must never change the case of the local-part when it receives a mail and hands it on to another mail server. Our mailserver (and pretty much every mailsystem connected to the internet) has case insensitive usernames (local-part).

So, regarding the whitelist question. I'm really surprised that a lower case mail address doesn't match the mixed case address. I'll have to check spamassassin to find out if this is expected behavior.

[carverrn]

So, regarding the whitelist question. I'm really surprised that a lower case mail address doesn't match the mixed case address. I'll have to check spamassassin to find out if this is expected behavior.

SpamAssassin isn't case sensitive. The lower case names in the Whitelist will match any case. I tested it before and just retested it and it works fine.

It should be noted that a change to the Whitelist might not be instant. There could be several minutes before SpamAssassin actually sees the updated Whitelist. With my most recent test, a test message sent immediately after changing the Whitelist did not result in a USER_IN_WHITELIST. All subsequent test messages did.

Regards,
Rich

[Gnome]

It may be that way for you but it's not the case for me as of last Friday. That setup is working for every incoming email I have in my white list sans the one with the capital letters for the account name.

If it's not case then I don't know what it is.

[Gnome]

Something changed since Thursday/Friday then as this now works now with no changes on my end.

[carverrn]

Glad to hear it's working now.

Rich

[mail2me]

Originally posted by carverrn
Here's what seems to work very well for me.

(1) For "Detect junk mail" select "Yes, reject if possible"

This allows Runbox to reject a message at the time it's being received,
if it is flagged as SPAM and if everyone receiving the message has agreed
to reject SPAM. I originally did use this option but the Linpro guys (the
ones hosting the Runbox mail servers) convinced me it's a good choice.

Does the Runbox "reject" really work?

I have tried sending the famous "ADV: Your Membership Exchange order -- Question to eBay seller g.r.a.p.e?" message minus the "ADV" ( since FastMail doesn't allow sending messages with it in the subject line) from various accounts to test this feature. The message definitely gets flagged as spam and filed to the Spam folder but it doesn't get rejected.

[jbs]

Originally posted by mail2me
Does the Runbox "reject" really work?

I have tried sending the famous "ADV: Your Membership Exchange order -- Question to eBay seller g.r.a.p.e?" message minus the "ADV" ( since FastMail doesn't allow sending messages with it in the subject line) from various accounts to test this feature. The message definitely gets flagged as spam and filed to the Spam folder but it doesn't get rejected.

I think I can answer this, it's a bit tricky but here's how I understand it. When a spam comes through for, say, 10 users, SA looks at it and determines if it is, in fact, spam. If it is, then the Runbox mail server has the option of rejecting the entire transaction from the sending mail server, but ONLY if every one of the recipients had selected Reject If Possible.

If anyone did not have Reject If Possible chosen, then Runbox is required by email protocol to accept it and deliver it to all recipients.

The strange thing in what you describe is, as I understand it, you should not be able to have simultaneously selected "Reject if Possible" and "File in XX folder". They are 2 different options.

So, for people who choose Reject if Possible, you need to set up a filter in your Manager to reject anything that meets your criteria of spam. So mine, for example, says any messsage whose header contains "X-Spam-Level: ****" should be filed as spam. This catches anything with a SA score above ~4 and moves it to my Spam folder.

Post here again if you have "Reject If Possible" selected, no other Spam filters that you've set up yourself, but Spam getting moved to a separate folder. I'd be interested in understanding it better.

What you've described (not being rejected, filed in folder) would make sense if the second of the 3 spam options is selected, "Detect, and file in XX".

In the mean time, hope this all makes sense.

--Jason

[mail2me]

Hi Jason,

Thanks for the reply.

These are some of my points.

1) I have currently selected "Yes, reject if possible". The "Yes, save to folder" is Trash but it is not selected. I also have a filter order -2 which reads:
Messages where: "Header" "contains" "X-Spam-Flag: YES" will be: "saved to folder" "Spam" as suggested above by Carverrn.

2) Previously I had also selected "Yes, reject if possible", but the "Yes, save to folder" was Spam but it was also not selected. However, the spam messages continued to file to "Spam" folder. Don't you think this is strange?

3) I sent out a sample message from both my FastMail account as well as my MailSnare account. They both had my correct email addresses as the "From" address. The messages were classified as spam by runbox, but were filed to folder instead of being rejected. Can you explain it?

4) Can you give me one example of how I can test the reject feature to see that it really works?

Thanks.

[carverrn]

How the "reject if possible" is suppose to work was described in this posting by Sigurdur from Linpro (the host for Runbox's mail servers):

http://www.emaildiscussions.com/...203#post160203

Based on that I would have expected your test SPAM to be rejected (assuming you only sent it to one Runbox address).

I think SpamAssassin is handled at the Linpro end so maybe Liz can ask them to comment on this.

Rich

[tore]

Originally posted by mail2me

1) I have currently selected "Yes, reject if possible". The "Yes, save to folder" is Trash but it is not selected. I also have a filter order -2 which reads:
Messages where: "Header" "contains" "X-Spam-Flag: YES" will be: "saved to folder" "Spam" as suggested above by Carverrn.

Actually, the "save to folder" setting is just a filter much like the one you've added manually. It doesn't check the headers, but a flag in the database, but it won't make a difference to you.

The folder in the "save to" setting is used even if you've selected the "reject if possible" option (I think so, at least), so you could just do that instead of adding the filter.

Do note the "if possible" part of "Reject if possible", though. For instance, if the mail is received from POP retreival or sent by another Runbox user from the web interface, this setting won't be used.

2) Previously I had also selected "Yes, reject if possible", but the "Yes, save to folder" was Spam but it was also not selected. However, the spam messages continued to file to "Spam" folder. Don't you think this is strange?

I suggested having the "reject" option as a check box (which then could be turned on/off if the "Yes, save to" radio box setting was selected) when we implemented it, and I still think that would have reflected the actual process better. But I don't meddle around in the webapp code at all, so.. shrug

3) I sent out a sample message from both my FastMail account as well as my MailSnare account. They both had my correct email addresses as the "From" address. The messages were classified as spam by runbox, but were filed to folder instead of being rejected. Can you explain it?

The reject feature was temporarily disabled a few days ago due to the load it inflicted. A new box -- bolivar -- was installed today to take over that particular task, and I expect to re-enable the feature sometime tomorrow. This new box should also pave the way for extended spam filtering functionality such as per-user bayesian databases, too.

4) Can you give me one example of how I can test the reject feature to see that it really works?

You could try sending yourself a mail with the eicar.com test-virus attached (from another system). It will be rejected in the exact same manner as a spam would have been. Or, you could wait until sometime tomorrow and send yourself a "spammy" e-mail, and hope that I've had time to set up SpamAssassin on the new box and make the MXes use it.

[jbs]

Sounds like that explains why it was not rejected -- that the feature was temporarily disabled. Otherwise, mail2me, I would expect that your spammy message, sent from outside Runbox to only one recipient at runbox should be rejected.

Tore- I did not think that the file to folder still worked even if the Reject if Possible option was selected. I agree with you, it should be possible to select both, the Reject should be a checkbox enabled as an option when the Detect and File is picked.

I have my personal filter set up because I thought I'd read in a posting that the file to folder would not work if the reject option is selected.

By the way, for mail2me and others, the reason I filter on the occurrence of "X-Spam-Level: ****" instead of just SpamFlag: Yes is that it allows you to tailor your spam tolerance. SA decides it's spam when the score is 5 or greater, but I was occasionally getting spam that scored a 4, 4.5 4.9 etc, so I lowered my "tolerance" to 4. You could also make it X-Spam-Level: ** which would lower it to 2 (and still would catch ***, ****, *****, etc), just depends on how aggressive you want to be.

--Jason

[mail2me]

Thanks tore for the reply. You have answered all the questions.

I will wait till the reject function is enabled before testing again.

[carverrn]

Hi tore!

This new box should also pave the way for extended spam filtering functionality such as per-user bayesian databases, too.

Speaking of spam filtering, could you jump over to this thread and address a question I had with the SpamAssassin "autolearn" values in the headers?

http://www.emaildiscussions.com/...threadid=21309

Thanks,
Rich