IMAP - every day more painful than the last

[robegusn]

I have used Runbox for a lot of years and my account is paid up to 2011. The Runbox email address I use has been given to friends and family and Runbox used to be a reliable service.

This morning however, after sitting and watching Thunderbird doing nothing for 15 minutes as it tries to use the IMAP server I am getting disheartened.

I could simply forward everything to an alternate service - and that is likely what I will do - but my preference would be to have Runbox fix their IMAP problem. It has been years of promises and months of "very soon".

As the operators of the service, you must be sick of people complaining. As tech people you must be frustrated trying to fix an un-ending problem. As end-users we are frustrated.

When you are in a hole, the first step is to stop digging. Trash what you have and migrate to a commercial mail server. You can get some very good and scalable products for less than $5000USD. I have a very hard time believing that you haven't spent the equivalent value in time and efforts already.

It isn't about giving up, it is about being pragmatic.

Robert

[Geir]

We appreciate your frustration and thank you for your suggestion and your continued subscription.

The IMAP performance problems are complicated because they are connected to the structure of the "database accelerated" Runbox system, which in theory is very efficient -- to the extent that the various interfaces (web, IMAP, etc) are.

While we think we can improve performance significantly by tuning the various software components already in use, we're also considering alternatives -- among other things a relocation and major hardware upgrade; an option that has just recently come up.

We will discuss our alternatives in depth with our sysadmins next week and will inform you when we reach a conclusion and/or result.

By the way, IMAP performance seems to vary greatly with the client in use. For instance, Windows Mail (despite its other disadvantages) seems to download/update much faster than Thunderbird.

- Geir

[robegusn]

Geir

There are many other mail systems that use databases. We both know of mail systems that use MySQL, Postgres or even Windows based systems.

Of course storing mail in a database increases the speed and reliability of a system. However, you have been trying for a very long time to make things work. I know it is hard to abandon something you have put your heart and soul in to, but sometimes you need to know when to bail out.

I have great admiration for your drive and commitment, but ask yourself the question "am I standing too close to see the problem?".

You do not need to get tied in to complex licensing issues to resolve the problems. There are a lot of open source alternatives, as well as commercial products that have licenses that are flexible enough to allow for customization. Yes there are some very good pieces within Runbox as a whole, but when you consider that these constant IMAP issues are alienating even your most loyal customers, it is time to change strategy.

Robert

Geir - please email me when you have a minute - thanks.

[adamevans]

I gotta agree with everything robegusn said. This is silly.

I've complained a lot about the IMAP. A lot. And it's deserved every bit of my complaining. The IMAP service here has been horrible for a huge length of time now.

But you know what? Amazingly, the last few days, the IMAP has actually been even worse. It was slow as molasses before. It's damn-near unresponsive for me at this point. I don't know what you guys are doing or not doing, but it's getting worse.

And like robegusn, I actually don't want to switch. I've been with Runbox for years. The service wasn't terrible at first, and I planned to use them pretty much indefinitely, and I invested all my personal and business contacts in my Runbox email.

Because of that, I've had to stick with Runbox through this garbage, because relocating everything from my Runbox email to a new email is a huge pain in the ***. I've already started doing it with a new more-reliable company, but I'm probably going to have to stay with Runbox another year just to make sure I didn't miss anything important.

It's insane. It's just insane that this is still going on. I've seen some bad company service, but this is above and beyond.

[robegusn]

There have been so many threads about the IMAP issues that it would appear that there has been little to no activity in regards to this thread.

Two years ago I actively encouraged everyone I worked with to check out Runbox. The whole idea of consolidating everything in to a single 10GB account made sense at so many levels. The price was right, the service promising and the functionality well suited to the purpose.

Now I have over 3GB of mail stored on the Runbox servers and the idea of moving to another service is not an attractive option, nor is it simple to do. The phrase "between a rock and a hard place" is an apt description.

This is not a technical problem anymore - it is a choice to not look at any other existing solutions. I know Geir is working hard to resolve the issue and that he is very proud of the system that has been built.

Geir - as you read this, you should be proud of Runbox. As a whole, everything is very well done, but there are some real, serious and increasingly frustrating issues. The number one issue without question is IMAP. What is the use of having 10GB of storage and keeping emails if the IMAP protocol is not meeting end-user expectations?

There are both inexpensive commercial and open source mail solutions that could be leveraged resolve the issues. Yes it means compromise, yes it means some re-architecture of the system, and yes it is not a preferred solution, but sometimes being pragmatic is the most logical choice.

Robert

[Geir]

Robert,

Thank you for the insights and your sincere advice. You are right in assuming that we have become attached to the service we've built over the past 8 years, but we have also become increasingly pragmatic. This entails that we are indeed considering alternatives to several aspects of the system; however, as you understand there are a number of compatibility and integration issues that complicate a replacement of a component such as the IMAP server.

There have also for a long time been various internal issues within the Runbox company that have only recently been resolved, and that indirectly affect the development and operation of the service. Because Runbox has experienced a lack of resources it still takes time to correct deficiencies and catch up with both hardware and software upgrades.

Replacing the email software in the Runbox system with a standard email solution would involve a great deal of work (and probably downtime), and it's not clear that such an undertaking would produce a better result as long as the underlying cause of the IMAP performance degradation is not fully established. This is closely tied to database server performance, which we will look at more closely with our sysadmins tomorrow when we have planned to start deploying the new IMAP server.

We appreciate your concern and the dialog you've initiated. If you have experience with, and can recommend, any specific email solutions that might fit well with Runbox we're certainly interested as we are currently looking at alternatives regarding both software, hardware, and hosting facilities.

- Geir

[robb1]

robeguson said > As the operators of the service, you must be sick of people complaining. As tech people you must be frustrated trying to fix an un-ending problem. As end-users we are frustrated. <

Amen to that. I was a loyal runbox customer as long as I stand it... and still log in to this forum every couple weeks hoping to see "happy" messages... and I will continue to do so at least until my subscription runs out next April.

But the last 6 months or so of sluggish IMAP performance finally got to me. I tested Gmail's IMAP and was stunned to see how fast IMAP could be. However Gmail has several oddities that I couldn't deal with like the "on behalf of" thing and the fact that they spy on me and most importantly that they'll never give a damn what a power user like me wants out of his email provider.

Sooo.. I settled on that other IMAP provider which isn't quite as fast as Gmail but it's pretty darn fast. But the webmail interface really sucks compared to Runbox and very little of it is as clean and intuitive as runbox. But it does work, so I moved all my stuff over. I will pop back in now and then to see how Runbox is doing.

In other words, just make it fast, and I will happily move my biz back to Runbox.

Robert

[liteswap]

Exactly so. What he said: only I won't come back having gone through the migration process to Fastmail I'm not doing it again. When my sub runs out on February 2009, that's it, I'm afraid.

Geir, I was a happy customer from 1999 and it was great but six months of sluggish to non-existent IMAP (timeouts and all that) was too much. Email for me is my professional communcations tool and cannot be compromised.

Yet the rest of the world seems to be able to offer an IMAP service that runs quickly under Thunderbird. The fact that Runbox seems unable even to diagnose the problem (as evidenced by your remarks earlier up this thread) suggest that my decision to bail three months ago was the right one.

[totopo]

"Because of that, I've had to stick with Runbox through this garbage, because relocating everything from my Runbox email to a new email is a huge pain in the ***. I've already started doing it with a new more-reliable company, but I'm probably going to have to stay with Runbox another year just to make sure I didn't miss anything important."

Same here.


But as soon as I find the solution I am gone, very slow service.

[robegusn]

Geir

My suggestion is that you look at Desknow - http://www.desknow.com/

I have no affiliation with the product, I do not receive any financial benefit, nor do I have any association with the company. All I can say is that the product works very well, has the slickest web interface I have ever seen, source code is available and they have a migration tool.

At the whopping cost of $1950USD, I suggest you take the cost of the software, the costs related to migration and hardware, divide that by the number of Runbox users and ask if they would be willing to front a one time cost to help mitigate any financial impact the purchase would have on your business.

I can not speak for everyone, but I would gladly pay a one time cost of $100, or even $200; to simply avoid the pain of losing my Runbox account and have a real, working and up to date email system.

Desknow runs on everything from Windows to Linux to BSD - it simply works.

Robert

[Geir]

Thanks for your suggestion. DeskNow is an interesting groupware server, but it doesn't appear to be open source (GPL)?

Runbox prefers open source software so that we are free to modify the code to suit our purposes. As you might know, Runbox' strategy is to combine open source components from different suppliers to provide a unique set of features with unique advantages (and, necessarily, disadvantages).

We've also looked at Zimbra, but it's difficult to estimate the hardware requirements of running another server with the current storage and processing capacity. Regardless, it would require converting the Runbox email storage structure -- something we are planning to do anyway to improve IMAP performance.

Another problem with DeskNow appears to be the licensing fee -- the prices listed are for a single server only.

By the way, we installed a new version of the new Courier IMAP server today. We have replaced some of the database queries with more efficient ones, and it seems to be performing better.

- Geir

[adamevans]

All right... This almost pains me to write this, BUT...

Did you guys do something to the 143 IMAP server?

It's still unacceptably slow, but there's been some kind of improvement in the last couple days. It's still really slow--annoying slow--but it's actually almost functional now.

Did you guys do something?

[marc_otten]

Did you guys do something to the 143 IMAP server?

Perhaps this is caused by the installation of the new Courier IMAP server? (see Geirs last post for more info).

--Marc

[robegusn]

I did not see any difference in performance yesterday, but this morning (July 25, Alberta - Canada), there seems to be a marked difference.

Robert

[adamevans]

Originally Posted by robegusn

I did not see any difference in performance yesterday, but this morning (July 25, Alberta - Canada), there seems to be a marked difference.

Robert

Yeah, there's definitely a difference.

I mean, I want to be clear. The IMAP is still unacceptably slow.

But there seems to have been some improvement. Which hasn't happened in, like, ever.