|
The Technical Zone... The Geeky forum... Use this forum to discuss technical aspects of email, from authentication protocols to encryption. |
|
Thread Tools |
29 Jun 2017, 04:18 PM | #1 | |
Member
Join Date: May 2010
Posts: 47
|
Mail forwarding: good or bad?
This is a continuation of a discussion from another thread. Some interesting technical issues were discussed there, with some points that I agree with. But I disagree with the opinion that mail forwarding is "wrong" (or "broken" as stated in the cited Richweb article).
Quote:
Let's look at the above-cited "Why Email Forwarding Is Broken" article from Richweb, and see why their article is broken: 1. The article characterizes legitimate use cases of forwarding as being "impersonation," which it isn't. As I mentioned, that characterization is supported neither by the RFCs nor by the users legitimately using the feature. Labeling it as "impersonation" seems to be an attempt by the authors at spin-doctoring (i.e., resorting to pejorative wording, instead of just relying on the merits of their technical argument). (The above is the main problem with the Richweb article, the rest are perhaps less significant.) 2. They state that SPF stands for "SenderPermitted From". It actually stands for "Sender Policy Framework." 3. In the "Rosalie" example (test.com forwards to hotmail.com), they said "test.com is now seen as a SPAMSOURCE" in the scenario described. That is inaccurate. In that scenario, while the IP address (and possibly hostname) of any server transmitting the message might be identified as a spam source (including the server that hosts the "test.com" domain, whose domain name is presumably that of the hosting provider), the name of the hosted domain ("test.com") would not be identified as such, because that domain name appears in the "To" address of the message, not the "From" address (it is the "From" addresses that get scrutinized in this kind of analysis). 4. The article is poorly formatted (has many words run together due to missing spaces). Although Richweb claims to have deep technical experience, the above 4 points detract from my impression of them as a source of expertise on this issue. (It's also a bit humorous that, at the bottom of the article, they claim over 17 years experience, which is apparently true... but they can claim a somewhat higher number if they've really been around since 1995!). Here's my own use case. I have my own domain (website and email) hosted on Unix-based web hosting provider. I have various configured address of the form alias@mydomain which all arrive at my webhost. There, the messages get stored into flat Unix text files, and also get forwarded to my account at FastMail (with different aliases directed to different folders in my FastMail account). There are various reasons why I do this. A. The webhost is good for Unix-level file actions (like grep, emacs, and any other ways of manipulating/archiving/organizing files that Unix allows, which is maximal flexibility), while FastMail is good as a mail UI. B. I like having my mail stored in two places automatically, in case I need to access something while one service is down or not accessible, or in case of accidental data loss (could be me pushing the wrong button) on either service. C. While FastMail is my day-to-day UI, the webhost is the best "first arrival" location because they do an SMTP-level reject of unknown aliases. I can't configure that on FastMail with folder-addressing and all the various folders I have there... even if I delete the folder, the mail still gets put into the Inbox. D. Hosting an email domain on FastMail would cost me extra, whereas email hosting for my configured domain is included in the price of my service on the webhost. The good news is: Forwarding works well for me. And with the minimum possible spam filtering, I rarely ever see any spam (probably because I'm careful to use throw-away aliases when signing up for online subscriptions, etc.). I'm not saying everyone should do exactly what I do, I'm just saying that my use case is legitimate, and it works. Saying that there's something "wrong" with my use case has no technical merit. Perhaps someone might say that it may not be working, that I may be missing emails and never know about them. Perhaps, but if my webhost got blacklisted for forwarding spam (such as in Spamhaus, Invaluement, etc.), then other customers who use email services on that webhost would be affected, and the webhosting company would have an incentive to get off the blacklist quickly. And perhaps that's what motivates some service providers to do what the Richweb article mentioned, "Many web hosts are now banning email forwarding to third party emailaccounts." But the solution isn't to outlaw forwarding. That's technically weak, because it incurs what's called "functional degradation," i.e., depriving users of legitimate functionality that used to work. Email providers (like Google and Microsoft) are now some of the richest companies in the world, so they ought to spend some money and brainpower coming up with solutions that don't incur functional degradation, not just punt on the problem and say "you can't forward anymore." I should point out that mailing lists are also forwarders. You can't just say, "no more mail groups, folks, sorry." I'm not on the payroll of those companies (not anymore, anyway), but I'm willing to help think about this problem for free! Let's see.... Well, one thing FastMail does that helps is that they actually have an account setting where I can specify one or more hosts that I receive forwarded mail from. The form says, "Use only if forwarding email to FastMail from another system. The Received headers of these hosts are parsed to find the true email source." It seems that they're actually thinking about this problem with the customer in mind! I think the ultimate solution has to do with reputation management. I think we're part of the way there, with clearinghouses such as Spamhaus and Invaluement. When an MTA receives a message, it should be able to assess a reputation depending first on who the immediate sender is, in combination with whatever upstream reputation assessment that sender passes along (ideally involving the reputation of the individual who sent the message, e.g., if I've been a paying customer for years with a service provider, they should attach a higher rating to my message than if I just signed up that day). Messages with low or unknown reputation at any link in the chain get rated lower, and then their messages might get throttled, filed separately, or rejected, depending on the rating, along with service provider policy and end-user preferences. Importantly, we should give the end user maximum choice over how they want dubious messages handled (e.g., FastMail has settings for what to do with messages at higher spam scores). At the technical level, I'm not sure if this should involve Authenticated SMTP, SMTP over SSL, more reputation clearinghouses, or something else (they're not paying me enough to come up with the complete solution!). But I think it's a combination of things that could evolve from this point (perhaps along the lines I suggested above). I'd change that last statement to, "All because some companies that carry weight in the email market aren't willing to do the due diligence and come up with a sound technical solution that keeps the user's legitimate needs at a priority." |
|
30 Jun 2017, 01:28 AM | #2 | |||||||||||
Essential Contributor
Join Date: Apr 2014
Posts: 399
Representative of:
MXRoute.com |
Quote:
Quote:
Quote:
It doesn't matter what an RFC says. You can't shout at your monitor to Google or Microsoft "You're violating RFC" and expect change. RFC doesn't make reality, and anyone is free to violate them at any time. Microsoft very frequently accepts emails with a positive return code and then never delivers them, no error or bounce. This violates RFC standards. They don't care. Quote:
Quote:
Quote:
1. Forwarding MTA might add the domain elsewhere in a header that recipient server doesn't recognize shouldn't be included in content filtering/learning. 2. SRS, which is vital for forwarding, will have the email sent from the recipient domain to pass SPF. Recognizing this at the end point is not a promise not to filter based on it, and some do. Whether intentional or not, it happens. Quote:
Quote:
Quote:
Quote:
Quote:
My reply was too long for this forum, you can read the rest here: https://paste.jarcloud.pw/ejolaqupot.sql |
|||||||||||
30 Jun 2017, 07:12 PM | #3 | ||||||||||
Member
Join Date: May 2010
Posts: 47
|
Quote:
Quote:
Second, the "range" is actually wider than you describe (that is, the range goes to far better than "intermittently functional"... even if it didn't where you worked). I do comparisons of messages received on the host that initially receives my mail vs. the host I forward to. I can tell if something failed to forward. I estimate that I've lost less than one message per 10,000 on a forward. The one I can think of was because the sender's domain was on the Invaluement blacklist. My webhost doesn't check that blacklist, so it accepted the message, but FastMail does, so they rejected the forwarded message (not because it was forwarded). Now, if I had done what you advocate and not do any forwarding, I'd never know that the sender was on the blacklist. Good thing I knew, because I was able to help them get off it (it was an arts school with nobody technical working there). Not trying to argue with you, just adding a data point that you might not have known existed (someone using forwarding successfully in the present age). Revising your sentence for accuracy, it would say, "ranging from nearly flawless to completely broken." Quote:
Quote:
What matters to me more is if the service providers I pay money to care about the standards. I have found that they do, and that's backed up in action by implementing systems that do, in fact, implement those standards competently (or so has been my experience with them). I'm not on any of the major providers you mentioned, so I don't care what they do other than allow me to get mail to/from their users via my service providers. I haven't had any trouble getting mail back and forth to people who are on those major services. I don't interact with the general public, mainly just people who I see in person or who I might talk with on the phone (and various businesses like my bank, etc.). So if they ever had trouble getting mail to me, I'd have heard from them about it by now. Apparently, the major providers implement the standards well enough that my service providers have a high success rate getting mail back and forth to them... even with the forwarding I'm doing. Again, not arguing, just adding a data point that you apparently thought could not exist. Quote:
Quote:
Also, if you're not relying on the RFCs, then I don't know what you're referring to when you say "SPF defines". And if you are relying on the RFCs, then your statement is inaccurate, in that they recognize forwarding as a legitimate action and recommend strategies for getting SPF to work in cases where forwarding is happening. And they do not refer to it as spoofing/impersonation. Maybe you should cite what definition of SPF you are referring to, because it's certainly not the one I found. I agree that how I feel about it is irrelevant, except when I make choices what service providers I give my money to. And I'm satisfied with the providers I have. Quote:
Quote:
That was just my experience. Yours may be different. Quote:
In any case, I can add another data point for you. I've been using my own domain in combination with the forwarding arrangement I described previously (which is very similar to the 'Rosalie' example) for 15 years. In all that time, even though I was a recipient of some amount of spam over that time, I never had the experience where my domain name got blacklisted as a result of appearing in the "To" header. Again, not disagreeing with you. Just adding a data point that shows that the system does work, at least on competently designed servers. Quote:
I'm no longer on the payroll of any company that's involved in email, so I probably won't show up there myself. I'm just an end user, hoping that the people who are paid to work on this stuff work out a solution that's in our best interests, not just punt on the problem and impose functional degradations on us all. Yes, it may be hard work, but that's what the 'E' in 'IETF' is for. It means to think and work constructively, not just give up. In my previous message, I pointed out how FastMail is looking at the problem constructively, and alluded to how further progress can be made. Being optimistic, my prediction is that the ultimate solution will be something along those lines. I stand by my earlier statements. |
||||||||||
3 Jul 2017, 06:45 AM | #4 |
Essential Contributor
Join Date: Apr 2014
Posts: 399
Representative of:
MXRoute.com |
Content filters are imperfect. Content filters are used to block sending servers. Forwarders cause you to be a sender of emails that trigger content filters. Different people will have varying degrees of impact on this due to matters outside of their control. Namely, who targets them for spam. That is something a mail provider has no control over, how many spam lists a customer's email address is on.
We can wax poetic about it all day, but those are hard facts and it doesn't matter how anyone feels about them. We can argue about what/why, but forwarders get you and other people blocked by major mail providers far more commonly than a lack of forwarders. Period. Cold, hard fact. It's unacceptable to allow one customer to harm another's quality of service. Individual anecdotal experiences do not compare to my experience. Not trying to be condescending, but it's like being an accountant and someone on the street walks up to you and tells you that you're not an accountant. That's about as annoyingly condescending as it gets for me. I'm either lying or you're wrong, period. I'm not lying. You do the math. Now I've given you every opportunity to understand what it's like, you refuse to accept it, that's your problem and no longer mine. I have 3,000-10,000 emails per hour to check up on every day, I don't have much time to explain why I'm not hallucinating or making up stories. If you're not interested in my insights, I'm just as happy to not give them. Last edited by jarland : 3 Jul 2017 at 07:05 AM. |
3 Jul 2017, 11:52 PM | #5 | |
Essential Contributor
Join Date: Jan 2017
Posts: 280
|
Quote:
That's not really fair. The large free email providers have implemented, or are about to implement, ARC (Authenticated Received Chain) which aims to fix the the forwarding problem. I think you are overstating the current problem, the reality is that experiences range from unreliable to just fine. |
|
4 Jul 2017, 04:22 AM | #6 | |
Essential Contributor
Join Date: Apr 2014
Posts: 399
Representative of:
MXRoute.com |
Quote:
In this thread: People who think "because it works for me" is any relevant indication of what happens at scale. |
|
4 Jul 2017, 09:48 PM | #7 | |
Essential Contributor
Join Date: Jan 2017
Posts: 280
|
Quote:
|
|
9 Jul 2017, 03:26 AM | #8 | ||||
Member
Join Date: May 2010
Posts: 47
|
Quote:
Quote:
Quote:
Quote:
I'm glad my service providers have attitudes that lead them to solving problems rather than getting mired in them. Good day. |
||||
25 Jul 2017, 07:47 AM | #9 |
Master of the @
Join Date: Feb 2017
Location: USA
Posts: 1,751
|
I'm not experiencing problems based on my own checking of various email addresses being forwarded to Gmail. Not losing any emails.
|