Go Back > Discussions about Email Services > Email Comments, Questions and Miscellaneous
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
Stay in touch wirelessly

Email Comments, Questions and Miscellaneous Share your opinion of the email service you're using. Post general email questions and discussions that don't fit elsewhere.

Thread Tools
Old 12 May 2017, 01:15 AM   #1
The "e" in e-mail
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,546
Pegasus developer news update

Update from David Harris, the developer of Pegasus. Pegasus is one of the oldest email clients on the Internet (since 1993).


I'm sure many people must be wondering if I'm still alive... I have to apologize for being so uncommunicative for so long - as I've remarked in other posts, for someone who has put most of his adult life into trying to help people to communicate, I'm pretty poor at doing it myself. I'd love to tell you why I find it so hard to do regular updates, but I can't, because I'm not really sure. It just seems to be a task I find unusually difficult: a lot of the time, it feels like I only have dry, technical things to report, and I guess I just don't imagine my community being too interested in that stuff. Still, there's no doubt I need to work on my communication skills.

My community of users have to be the most patient people in the world: each week I get a few messages asking me how Pegasus Mail v5 and Mercury v5 are going, but they're never angry or impatient, always encouraging - and I have to say I find that amazing... I've been saying that both programs were "near" for almost as long as I can remember, yet my community seems to be willing to accommodate delay after delay with a steady good grace I find quite humbling. I hope, in this posting, to give you an idea of why these releases are taking so very long.

To start with, you'll need some background.

The first version of Pegasus Mail for Windows to send a message appeared back in 1993 - that's twenty-four years ago. Since that time, I calculate there have been fifty-six releases in various forms, each release building on the ones before it in some way or other. That's a lot of releases, each one representing many, many changes to the source code I write to produce the program. Over the years, that source code has become ever larger and more complex; furthermore, because I'm human, my style has grown and evolved over the years, and the way I wrote things back in 1993 is almost invariably not the way I would do them now, so a lot of the code exists in a mixture of styles that makes maintenance very difficult.

By 2010, it had become absolutely clear to me that I could no longer reliably maintain the Pegasus Mail source code as it stood - it had become too large and had too much history and compromise in it... Add that to the fact that Mercury had also grown large and unwieldy, and I realized that I was going to have to find some way to reorganize and unify the programs - come up with common internal components so that I was maintaining less code, and the code I was having to maintain was more modern, more consistent, more streamlined.

At the time, I thought I could do this as a gradual process - replace a piece here, a piece there, and the programs would progressively converge on a new, maintainable standard... But I was wrong - or at best, naive. I would start working on modernizing one part of the program, only to find that if I were to do the work properly, I could salvage almost none of the old code - I would have to write it anew. Then I'd find that rewriting that part of the program couldn't proceed until I'd rewritten another part of the program on which it depended, so the process would start once more - and so on and on, over and over again.

Although it certainly wasn't my original intention, what I actually ended up doing was rewriting almost the entire program from scratch: this wasn't a process of modernization, as I had initially planned, but a process of writing a completely new program using an older one as a model.

It can be hard to get perspective on this process - on how large the work really is... But over time, I've come to look at what I do in terms of units I call "WaPs" - short for "War and Peace": Tolstoy's "War and Peace" (Oxford World Classics edition) has about 71,500 lines of text, so I defined 71,500 lines as 1 WaP, and use that to measure the size of the work I do. At the moment, the totals are roughly as follows:

Pegasus Mail 227,428 lines 3.181 WaP
Mercury 169,163 lines 2.366 WaP
Setup and Tools 154,321 lines 2.158 WaP
Manuals and Docs 96,210 lines 1.345 WaP
Totals: 647,122 lines 9.050 WaP

That means that the code I am working on is currently roughly the size of nine copies of War and Peace... That's a lot of lines. I further estimate that in the last four years, I have updated, added or removed around 200,000 of those lines of code (2.797 WaP), and although that's an estimate, it's on the low side if anything. The cold reality of these numbers is that the combined bulk of Pegasus Mail and Mercury is now beyond the ability of a single programmer to maintain and improve unless ways can be found to streamline the process as much as possible, and it's mainly this that I have been focusing on for the last few years. I have no intention of giving up on these programs that have been part of my life for so long, but I simply have to find ways of making them manageable.

So what have I actually done so far? Here's a potted summary, by no means complete:

Completely new MIME and RFC2822 message parsers (the MIME parser in particular is, I believe, as good as any in the entire industry).
Completely new RFC2822 message generator.
Completely new encryption support, including S/MIME support and support for encrypted databases, folders, contacts and lists.
Completely new contact manager (a kind of mega-addressbook that I personally think will be the most game-changing component in v5).
Totally overhauled user interface elements, including support for simple user-defined style changes (also known as "skinning").
Totally overhauled low-level tool sets - for example, a string library that can handle dynamic strings up to 1Gb or larger in size.
Completely new TCP/IP network layer with OpenSSL support (this component has already found its way into current versions of Pegasus Mail and Mercury).
Totally new console objects that allow the separation of the Mercury service processes from its user interface (this, in fact, was finished quite recently).
A totally new object-oriented framework to tie everything together.
A completely rewritten low-level message store.
The last two are so important that they deserve some elaboration: the object framework (which I call "OIF" internally) is the basis for all the new code, and will allow third-party add-ons to be first-class citizens, with the same access to the internals of Pegasus Mail and Mercury as the programs themselves: this desire to open up the "guts" of the programs in a consistent, effective manner that made it easy for people other than me to create powerful new features for them was a major goal of the redevelopment, and required enormous effort, although I believe the results are really good.

The new Message Store is an example of how difficult some of this stuff can end up being; I have been working on it for several years and in that time have ended up scrapping what I had done and restarting twice. The Message Store is the low-level code that implements the folders and messages you work with on a day-to-day basis, and I wanted something that provided a clear way forward to future capabilities without sacrificing compatibility with existing data; getting the right balance between past and future makes this process unusually difficult - had I been able to scrap everything and simply start from scratch, it would have been a far simpler process, but I value data compatibility above almost everything else: it is a matter of pride to me that a folder created by the DOS version of Pegasus Mail in 1990 is still readable by the latest Windows version in 2017, and will remain readable by v5, even though newer and vastly better folder formats will also be available.

Hindsight, as the old saying goes, has 20-20 vision, and in hindsight, I would have to say that I could have handled the process of updating and modernizing my programs considerably better than I have. Had I realized when I started what I was getting myself into, I would certainly have tried to find a different strategy, some way of making the process cleaner, more compartmentalized, and more inclusive of my user community... But in the end, the code would still have had to be written, and it was always going to take a long time no matter how I approached it.

I hope this rather over-wordy update (roughly 0.002 WaP in size) offers at least some explanation why the process of bringing out the next generation versions of Pegasus Mail and Mercury is taking such a long time: in offering it, I would also like to offer my thanks to my community of users, testers and supporters for their considerable patience and encouragement, and my hope that they will stick with me for just a little longer. I really do feel like there's light at the end now.

All my very best to you all,
Berenburger is offline   Reply With Quote

Old 13 May 2017, 06:51 PM   #2
Join Date: May 2017
Location: Emergency temporary account of ROBERT.BAK
Posts: 36
It's to be hoped that PMail 5 arrives sometime soon; it's incompatible with Win10, and for obvious reasons running it in a sandboxed VM is not a workaround for that problem.
evilquoll is offline   Reply With Quote
Old 13 May 2017, 09:04 PM   #3
Essential Contributor
Join Date: Jun 2009
Posts: 373
To: Berenburger

Pegasus and Windows Commander are the only two Windows programs I have missed since I stopped doing Windows ten years ago.

I tried running them with Wine on Linux, but it wasn't the same.

Other than those two programs and with all due respect to Windows users, I haven't regretted my decision to leave Windows for Linux.

Jacinto is offline   Reply With Quote
Old 19 Jun 2018, 06:18 PM   #4
The "e" in e-mail
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,546
Building Blocks, June 2018

New post from David Harris about the Pegasus development.


The release of WinPMail v4.73 gives me an opportunity to describe a little of the process that goes into developing a program like this - in particular, I'd like to provide a little insight into the new help system in v4.73, which is the first Pegasus Mail Version 5 feature to make it to release.

The new help system became a necessity when Microsoft retired the old WinHelp help format a few years ago, because they simply didn't replace it with anything suitable for developers like me who had an investment in it. The HTMLHelp program they provided was adequate for many developers, but suffered from a crucial flaw: because it depended on Internet Explorer for its HTML rendering, it could not load help files on shared volumes (IE won't do this, supposedly for security reasons). There's absolutely nothing worse than being stressed by a problem with a program, pulling up its help system, and seeing "Navigation Cancelled" instead of a help page, yet this is what happened if you tried to open an HTMLHelp help file that was installed on a server or NAS shared volume. There were mitigations for the problem, but they were poorly documented and technically demanding for anyone who wasn't a trained sysadmin.

So, after looking around to see what else was available and finding almost nothing, I began rolling my own using components I already had for Pegasus Mail. It was a long job - I estimate that developing the help compiler, the help display module, and the user interface for the help system took me about 400 hours over a period of years, and I know for a fact (having done it recently) that converting the old RTF source I used for the Pegasus Mail WinHelp help file to HTML for use in the new system took me 150 hours (yes, I was counting it was a HORRIBLE job).

"But it's just a help system!" I can hear you saying; "it's just a little text and a window to display it in!". Well, the WinPMail help pages consist of more than 250 separate help files, all cross-indexed and linked, each formatted as consistently as possible: that's roughly 20,000 lines, or (for those of you who can recall my last developer update) about a third of a War-And-Peace, and that's just the basic text, without considering the markup, indexing and table of contents generation. The source code for the help compiler and the display module make up about another 15,000 lines of code - that's just the reality of working with a format as rich and complicated as HTML.

All of this just to replace something that Microsoft decided to remove because they couldn't be bothered keeping it up to date... That said, though, the result is pretty good, and I now have a help system that doesn't depend on anything outside my control, and that I can use in both Pegasus Mail and Mercury I have only one piece of code to maintain, completely self-contained, unaffected by any changes to the programs that host it. And it is this that is really the goal of this update - to describe a process I am now applying to all my new development: code that stands on its own, can be used in either system, is available to extension developers, and requires minimal support in the long term.

This approach to developing things as building blocks gives flow-on benefits that can be guite interesting so, as an example, the module I'm just finishing now that implements full S/MIME capability will allow Pegasus Mail to send and receive a full range of modern encrypted data, but will also allow Mercury to send out its list mail with digital signatures as proof of origin; in a similar fashion, the new Pegasus Mail Contact Manager (the replacement for the aging addressbook system) could easily serve as the basis for a Mercury vCard server and there are countless other similar examples where having the one self-contained module will allow both programs to benefit: I'm expecting that there will even be crossovers I haven't anticipated yet.

Interesting times indeed.

All my very best to you all,

-- David --
Berenburger is offline   Reply With Quote
Old 23 Jul 2019, 07:33 AM   #5
The "e" in e-mail
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,546
The Eyes Have It

Somewhat worrying news about David Harris.


In January this year, while I was taking a brief holiday with some friends in Christchurch (a city about 350km north of Dunedin, where I live), I ended up being admitted to hospital for emergency eye surgery on a detached retina. It was probably just as well that my friends are both doctors, since it meant that the condition was caught early.

The surgery was successful, but since that time I have had almost constant problems with my vision, partially as an after-effect of the operation, but mostly because of a rapidly-growing cataract that has formed on the lens of my left eye: for the last three months, I have had almost no useful vision in that eye. I am scheduled for priority cataract surgery to correct the problems, after which I should be back to normal (or even, they tell me, a little better than before the initial problem), but at this stage the best estimate the hospital can give me for when it will happen is "sometime in the next four months".

The effect this series of issues has had on my life, and on Pegasus Mail and Mercury, is quite considerable. My right eye has always been weaker than my left eye, but it now has to handle the entire burden of my vision. What this means for my user community is that I can only do screen work for about half an hour at a time before getting eye strain: I'm still working, but at a reduced rate, and my ability to do things like answer mail is heavily reduced.

Warning: raw emotional outpouring alert!
It's frustrating. So mind-bendingly frustrating! Some days it makes me so desperate I could almost cry.
End of emotional boilover

Now that I've got that off my chest, let me give you a small progress update. Mercury v5 is very near to going into beta - I just have to finish the mailing list manager overhaul that is required to support the new split interface and it will be ready. I do not expect a long beta period for v5 - I have done huge amounts of testing as I've worked on it (I had to - just about every source module in the program has changed significantly).

WinPMail has been lagging behind a bit as I put my efforts into Mercury, but I would rate progress as good, and believe I should be on target for rough public betas before the end of the year. WinPMail needs fewer structural changes than Mercury, but far more modernization, and there's much more of it.

I will put out another developer update soon laying out some of the changes you can expect to see in the new versions of both programs, but for now, I'm afraid, my half hour is up, and I'm squinting so hard at the screen I can feel my eyelids seizing up.

All the best!

-- David --
Berenburger is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump

All times are GMT +9. The time now is 11:48 PM.


Copyright 1998-2013. All Rights Reserved. Privacy Policy