|
Runbox Forum Everything related to Runbox should go here: suggestions, comments, complaints, questions, technical issues, etc. |
|
Thread Tools |
30 Jan 2004, 06:00 AM | #1 |
Intergalactic Postmaster
Join Date: Jan 2002
Location: Chicago, IL
Posts: 5,606
Representative of:
Runbox.com |
How To Improve SPAM Filtering using Runbox Filters
2005-07-15: This information hasn't been updated since Runbox implemented the trainable DSPAM filter. But much of the information is still relevant.
A blocklist doesn't do much good for most SPAM since SPAM doesn't usually have a valid "From" address nor use the same "From" address everytime. Here's what seems to work very well for me. (1) For "Detect junk mail" select "Yes, reject if possible" I no longer recommend selecting this option. See below for details. This allows Runbox to reject a message at the time it's being received, if it is flagged as SPAM and if everyone receiving the message has agreed to reject SPAM. I originally didn't use this option but the Linpro guys (the ones hosting the Runbox mail servers) convinced me it's a good choice. 2005-07-15: After using "reject if possible" for a while I decided it's not a feature worth using. It can cause more problems than it solves. First, if you receive messages from groups like Yahoo Groups or Google Groups it can bounce spam messages back to Yahoo or Google which will cause your address to be marked as "bouncing" and you'll stop getting mail from your groups until you reactivate your address. Second, since a lot of spam is bulk mailed to many users at the same domain there's a good chance that for a given spam mail at least one user will not have selected this option so for that message it's disabled for everyone. Finally, it's probably more likely to bounce a valid message sent just to you that was incorrectly flagged as spam and you'd never know it. (2) Create a "Spam" folder and a filter defined as: Order: -2 Messages where: "Header" "contains" "X-Spam-Flag: YES" will be: "saved to folder" "Spam" By selecting "reject if possible" for the "Detect junk mail" option, we have also disabled the ability to move SPAM to a Spam folder. This defines our own Spam Filter that will do the same thing the original Spam Filter did. If you want your Spam Filter to be more restrictive, like have a cutoff of 4 instead of the default of 5, you can define your filter like this: Order: -2 Messages where: "Header" "contains" "X-Spam-Level: ****" will be: "saved to folder" "Spam" For this filter, any message with a score greater than 4 (e.g. 4.1) will be moved to the Spam folder. (3) Create a "NotInWhiteList" folder and a filter defined as: Order: 999 Messages where: "Header" "doesn't contain" "USER_IN_WHITELIST" will be: "saved to folder" "NotInWhiteList" This should be your very last filter. If a message makes it all the way to this filter, it's next stop would be the Inbox. This filter will make sure that the message is from someone in your Whitelist, if not, it moves it to the "NotInWhileList" folder. Initially you will have messages in your "NotInWhiteList" folder that you either want to add to your Whitelist or create a filter to move it to a folder (could even be the Inbox). You will also have SPAM that wasn't caught by SpamAssassin. Eventually, as you update your Whitelist and filters, the "NotInWhiteList" folder will contain mostly uncaught SPAM. (4) Messages over 250K in size are not scanned by SpamAssassin. The reason being that mail that large is unlikely to be SPAM. Unless these messages are moved to a folder by a filter, they would end up in the "NotInWhiteList" folder because they will never contain the "X-Spam-Status" header for the "USER_IN_WHITELIST" check. To avoid this you can add the following filter: Order: 998 Messages where: "Header" "doesn't contain" "X-Spam-Status" will be: "saved to folder" "Inbox" This moves ANY message that hasn't been scanned by SpamAssassin to the Inbox. Or you could create a NotSpamChecked folder and move it there. (5) Add addresses of people/places you expect "ham" (not spam) email from to the Spam Filter whitelist. When SpamAssassin checks a message it will check the whilelist for the "From" address. If it finds the address, it will add "USER_IN_WHITELIST" to the "X-Spam-Status" header in the message. (6) Add filters for Mailing Lists and Groups. For mailing lists or groups (e.g. Yahoo Groups) that use the senders real email in the "From" address instead of the list/group email address, setup filters to move these messages to the proper folder. If they use the list email address in the "From" address you could White List it instead but you still might want to have a filter to move them to a folder. (7) Add other filters as needed/wanted. Messages are checked by SpamAssassin as they arrive at the Runbox servers. However, messages flagged as *SPAM* are not filtered until your filters are processed. The Spam Filter runs at an order value of "-2". If you want your filters to run BEFORE the Spam Filter, allowing you to handle ALL messages, your filter order values must be less than -2 (-3 through -99). If you want your filters to run AFTER the Spam Filter, allowing flagged SPAM to be filtered, your filter order values must be greater than -2 (-1 through 999). CAUTION: You can have multiple filters with the same order value. Within the same order value, filters are processed in the order in which the entry was first added to the database. The filters page will show the filters in the order processed. It may seem a bit complicated but it's working really well for me. Regards, Rich Last edited by carverrn : 16 Jul 2005 at 01:00 AM. |
30 Jan 2004, 07:59 AM | #2 |
Essential Contributor
Join Date: Oct 2003
Posts: 455
|
Rich,
Thanks for an excellent set of instructions. I've dodged spam thus far but I think I will start building up my whitelist just in case. Along those lines I'm wondering whether there is an easy way to add a lot of addresses to the whitelist. I'd love to be able totake my contact manager software, dump out a file of all the email addresses and add them to my whitelist. Doing so I'd have 99% of my wanted email already flagged for whitelist. I tried using the RBTB2 and entering multiple addresses in the input field, separated by spaces, commas or semicolons but could only get it to work with one at a time. Is that the way it works? Along similar lines, I know your toolbar has an excellent tool for mass-uploading to one's address book . . . does being in the address book make a contact automatically count as whitelisted? If not, then can I respectfully suggest to the Runbox team (or the Toolbar "team" ;-) a mechanism for mass uploading of whitelist names? --Jason |
30 Jan 2004, 12:13 PM | #3 | |||
Intergalactic Postmaster
Join Date: Jan 2002
Location: Chicago, IL
Posts: 5,606
Representative of:
Runbox.com |
Hi Jason,
Quote:
Quote:
It currently only handles one address at a time. I could take a look at handling multiple addresses. Quote:
Regards, Rich |
|||
31 Jan 2004, 12:34 AM | #4 | |
Essential Contributor
Join Date: Oct 2003
Posts: 455
|
Quote:
Well, I hate to correct you in public like this, but you were wrong when you said there is a way 'kind of'. In fact, your bookmarklet (surprise surprise) is PERFECT!! Not only did it allow me to wasily and quickly add 500 email addresses to my whitelist, but it also id'ed for me a dozen or so cases where I had improperly formatted email addresses in my Palm Pilot! The process was relatively simple. I exported the email field of my Palm address book to a CSV file, opened in excel, then copied into notepad and back into excel (this split records where I had multiple email addresses separated by carriage returns into separate cells in Excel. Then I split data to columns using commas as delimiters, so that any fields where I had multiple emails separate by commas would go into separate cells. I pasted the cells that were now in columns B, C, etc into column A and sorted alphabetically. I then pasted them into your form (about 100 at a time) and voila! Just tested it with an email from myself (fortunately I'm on my whitelist -- it's obviously not that exclusive) and saw the beloved USER_IN_WHITELIST flag. I've never used this before because of the difficulty in adding addresses one at a time, and a concern that if I only added them from emails I got, I would slways be getting emails in my "NotInWhitelist" folder from that person you'd not heard from in a long while. Even though Spam's not been a problem for me yet at this new address, I'm going to modify my growing system of filters to mimic what you've described above, and start to build a good solid pre-emptive system for dealing with spam. Thanks again, Rich! You rock!! --Jason |
|
31 Jan 2004, 01:47 AM | #5 |
The "e" in e-mail
Join Date: Jul 2001
Location: Los Angeles,CA
Posts: 4,652
Representative of:
Runbox.com |
Just FYI, I'm working on incorporating Rich's excellent "scheme" in the help pages, just have to find a way that doesn't send the insecure users into spasms of panic.
Also, the current placing of the Toolbar and bookmarklets links is way too hidden away, I think (in the Help pages, under email add-ons). Liz |
31 Jan 2004, 02:14 AM | #6 |
Senior Member
Join Date: Dec 2003
Posts: 134
|
that would be great - but please, make it 'idiot-proof' so even I can undestand it
|
31 Jan 2004, 02:16 AM | #7 |
The "e" in e-mail
Join Date: Jul 2001
Location: Los Angeles,CA
Posts: 4,652
Representative of:
Runbox.com |
If I understand it, you will too.
Liz |
31 Jan 2004, 03:37 AM | #8 |
Intergalactic Postmaster
Join Date: Jan 2002
Location: Chicago, IL
Posts: 5,606
Representative of:
Runbox.com |
For the Spam Filtering, I know the process sounds a bit complicated although it's not too bad. I tried to add a little explanation of "why" it was being added but it might be easier for some if you just say "trust me, it will work" and just show the filters to add. Some better formatting via HTML and aa few screen shots might make it easier to follow too.
For the Whilelist editing, I said "kind of" because it would be a lot easier if it could import CSV files directly. I could probably do this with the import stuff form the Import Addressbook bookmarklet. I should probably write an Export Addressbook too then. I guess it's time to work on bookmarklets again. Rich |
31 Mar 2004, 04:20 PM | #9 |
Member
Join Date: Mar 2004
Posts: 76
|
i was wondering if anybody have time to make a visual repersentation of what carvern is talkin about. Not a lot of people would understand and i admit even tho im an IT major, i had trouble understand where is where and to input what.
|
31 Mar 2004, 11:56 PM | #10 |
The "e" in e-mail
Join Date: Jul 2001
Location: Los Angeles,CA
Posts: 4,652
Representative of:
Runbox.com |
Not a bad idea...if anyone would like to set up a dummy account with dummy contacts and make some screenshots with text a la the POP/IMAP setups guides (JPEG please), they will be rewarded. Do mail us first though, in case more than one person volunteers.
Liz |
1 Apr 2004, 04:26 AM | #11 |
Member
Join Date: Oct 2003
Location: Philadelphia, USA
Posts: 92
|
Hi, problem with setting these filters around case sensitivity of the in the email address.
My work address gets sent as: Joseph.Cairns@ibx.com But on the white list I'm only able to put: joseph.cairns@ibx.com There's no way for me to enter anything with upper case, so unfortunately, the server isn't putting the "USER_IN_WHITELIST" tag on this header. The process should be converting everything to lower case before it compares addresses but it doesn't seem to be doing this. Thanks! |
1 Apr 2004, 04:30 AM | #12 |
Guest
Posts: n/a
|
You might want to reformat your address so it doesn't get spammed (userATdomain instead of realuser@realdomain). Also, you don't need to put your username, enough to put your UserName for the example.
|
1 Apr 2004, 04:40 AM | #13 |
Member
Join Date: Oct 2003
Location: Philadelphia, USA
Posts: 92
|
How my email admins at work set up my address' format is outside of my control.
This is coming from my worlk server with the capitals but I'm unable to enter an address with capitals in the white list. Edit Note: regardless email addresses are not case sensitive and User@xyz.com should match user@xyz.com. |
1 Apr 2004, 04:45 AM | #14 |
Guest
Posts: n/a
|
according to the RFC's, the localpart is case sensitive.
I ment how you write your address in your post. (write supportATrunbox instead of support@runbox). |
1 Apr 2004, 04:50 AM | #15 | |
Member
Join Date: Oct 2003
Location: Philadelphia, USA
Posts: 92
|
Ahh gotcha have a million things going on here at once and my head is spinning.
My address at work is send only so no worries there, but I'll keep it in mind for future reference, thanks! Quote:
|
|
Thread Tools | |
|
|