[Berenburger]

Update from David Harris, the developer of Pegasus. Pegasus is one of the oldest email clients on the Internet (since 1993).

Source: http://www.pmail.com/devnews.htm

I'm sure many people must be wondering if I'm still alive... I have to apologize for being so uncommunicative for so long - as I've remarked in other posts, for someone who has put most of his adult life into trying to help people to communicate, I'm pretty poor at doing it myself. I'd love to tell you why I find it so hard to do regular updates, but I can't, because I'm not really sure. It just seems to be a task I find unusually difficult: a lot of the time, it feels like I only have dry, technical things to report, and I guess I just don't imagine my community being too interested in that stuff. Still, there's no doubt I need to work on my communication skills.

My community of users have to be the most patient people in the world: each week I get a few messages asking me how Pegasus Mail v5 and Mercury v5 are going, but they're never angry or impatient, always encouraging - and I have to say I find that amazing... I've been saying that both programs were "near" for almost as long as I can remember, yet my community seems to be willing to accommodate delay after delay with a steady good grace I find quite humbling. I hope, in this posting, to give you an idea of why these releases are taking so very long.

To start with, you'll need some background.

The first version of Pegasus Mail for Windows to send a message appeared back in 1993 - that's twenty-four years ago. Since that time, I calculate there have been fifty-six releases in various forms, each release building on the ones before it in some way or other. That's a lot of releases, each one representing many, many changes to the source code I write to produce the program. Over the years, that source code has become ever larger and more complex; furthermore, because I'm human, my style has grown and evolved over the years, and the way I wrote things back in 1993 is almost invariably not the way I would do them now, so a lot of the code exists in a mixture of styles that makes maintenance very difficult.

By 2010, it had become absolutely clear to me that I could no longer reliably maintain the Pegasus Mail source code as it stood - it had become too large and had too much history and compromise in it... Add that to the fact that Mercury had also grown large and unwieldy, and I realized that I was going to have to find some way to reorganize and unify the programs - come up with common internal components so that I was maintaining less code, and the code I was having to maintain was more modern, more consistent, more streamlined.

At the time, I thought I could do this as a gradual process - replace a piece here, a piece there, and the programs would progressively converge on a new, maintainable standard... But I was wrong - or at best, naive. I would start working on modernizing one part of the program, only to find that if I were to do the work properly, I could salvage almost none of the old code - I would have to write it anew. Then I'd find that rewriting that part of the program couldn't proceed until I'd rewritten another part of the program on which it depended, so the process would start once more - and so on and on, over and over again.

Although it certainly wasn't my original intention, what I actually ended up doing was rewriting almost the entire program from scratch: this wasn't a process of modernization, as I had initially planned, but a process of writing a completely new program using an older one as a model.

It can be hard to get perspective on this process - on how large the work really is... But over time, I've come to look at what I do in terms of units I call "WaPs" - short for "War and Peace": Tolstoy's "War and Peace" (Oxford World Classics edition) has about 71,500 lines of text, so I defined 71,500 lines as 1 WaP, and use that to measure the size of the work I do. At the moment, the totals are roughly as follows:

Pegasus Mail 227,428 lines 3.181 WaP
Mercury 169,163 lines 2.366 WaP
Setup and Tools 154,321 lines 2.158 WaP
Manuals and Docs 96,210 lines 1.345 WaP
Totals: 647,122 lines 9.050 WaP

That means that the code I am working on is currently roughly the size of nine copies of War and Peace... That's a lot of lines. I further estimate that in the last four years, I have updated, added or removed around 200,000 of those lines of code (2.797 WaP), and although that's an estimate, it's on the low side if anything. The cold reality of these numbers is that the combined bulk of Pegasus Mail and Mercury is now beyond the ability of a single programmer to maintain and improve unless ways can be found to streamline the process as much as possible, and it's mainly this that I have been focusing on for the last few years. I have no intention of giving up on these programs that have been part of my life for so long, but I simply have to find ways of making them manageable.

So what have I actually done so far? Here's a potted summary, by no means complete:

Completely new MIME and RFC2822 message parsers (the MIME parser in particular is, I believe, as good as any in the entire industry).
Completely new RFC2822 message generator.
Completely new encryption support, including S/MIME support and support for encrypted databases, folders, contacts and lists.
Completely new contact manager (a kind of mega-addressbook that I personally think will be the most game-changing component in v5).
Totally overhauled user interface elements, including support for simple user-defined style changes (also known as "skinning").
Totally overhauled low-level tool sets - for example, a string library that can handle dynamic strings up to 1Gb or larger in size.
Completely new TCP/IP network layer with OpenSSL support (this component has already found its way into current versions of Pegasus Mail and Mercury).
Totally new console objects that allow the separation of the Mercury service processes from its user interface (this, in fact, was finished quite recently).
A totally new object-oriented framework to tie everything together.
A completely rewritten low-level message store.
The last two are so important that they deserve some elaboration: the object framework (which I call "OIF" internally) is the basis for all the new code, and will allow third-party add-ons to be first-class citizens, with the same access to the internals of Pegasus Mail and Mercury as the programs themselves: this desire to open up the "guts" of the programs in a consistent, effective manner that made it easy for people other than me to create powerful new features for them was a major goal of the redevelopment, and required enormous effort, although I believe the results are really good.

The new Message Store is an example of how difficult some of this stuff can end up being; I have been working on it for several years and in that time have ended up scrapping what I had done and restarting twice. The Message Store is the low-level code that implements the folders and messages you work with on a day-to-day basis, and I wanted something that provided a clear way forward to future capabilities without sacrificing compatibility with existing data; getting the right balance between past and future makes this process unusually difficult - had I been able to scrap everything and simply start from scratch, it would have been a far simpler process, but I value data compatibility above almost everything else: it is a matter of pride to me that a folder created by the DOS version of Pegasus Mail in 1990 is still readable by the latest Windows version in 2017, and will remain readable by v5, even though newer and vastly better folder formats will also be available.

Hindsight, as the old saying goes, has 20-20 vision, and in hindsight, I would have to say that I could have handled the process of updating and modernizing my programs considerably better than I have. Had I realized when I started what I was getting myself into, I would certainly have tried to find a different strategy, some way of making the process cleaner, more compartmentalized, and more inclusive of my user community... But in the end, the code would still have had to be written, and it was always going to take a long time no matter how I approached it.

I hope this rather over-wordy update (roughly 0.002 WaP in size) offers at least some explanation why the process of bringing out the next generation versions of Pegasus Mail and Mercury is taking such a long time: in offering it, I would also like to offer my thanks to my community of users, testers and supporters for their considerable patience and encouragement, and my hope that they will stick with me for just a little longer. I really do feel like there's light at the end now.

All my very best to you all,