EmailDiscussions.com  

Go Back   EmailDiscussions.com > Discussions about Email Services > Email Comments, Questions and Miscellaneous
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
Stay in touch wirelessly

Email Comments, Questions and Miscellaneous Share your opinion of the email service you're using. Post general email questions and discussions that don't fit elsewhere.

Reply
 
Thread Tools
Old 12 May 2017, 01:15 AM   #1
Berenburger
The "e" in e-mail
 
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,894
Pegasus developer news update

Update from David Harris, the developer of Pegasus. Pegasus is one of the oldest email clients on the Internet (since 1993).

Source: http://www.pmail.com/devnews.htm

Quote:
I'm sure many people must be wondering if I'm still alive... I have to apologize for being so uncommunicative for so long - as I've remarked in other posts, for someone who has put most of his adult life into trying to help people to communicate, I'm pretty poor at doing it myself. I'd love to tell you why I find it so hard to do regular updates, but I can't, because I'm not really sure. It just seems to be a task I find unusually difficult: a lot of the time, it feels like I only have dry, technical things to report, and I guess I just don't imagine my community being too interested in that stuff. Still, there's no doubt I need to work on my communication skills.

My community of users have to be the most patient people in the world: each week I get a few messages asking me how Pegasus Mail v5 and Mercury v5 are going, but they're never angry or impatient, always encouraging - and I have to say I find that amazing... I've been saying that both programs were "near" for almost as long as I can remember, yet my community seems to be willing to accommodate delay after delay with a steady good grace I find quite humbling. I hope, in this posting, to give you an idea of why these releases are taking so very long.

To start with, you'll need some background.

The first version of Pegasus Mail for Windows to send a message appeared back in 1993 - that's twenty-four years ago. Since that time, I calculate there have been fifty-six releases in various forms, each release building on the ones before it in some way or other. That's a lot of releases, each one representing many, many changes to the source code I write to produce the program. Over the years, that source code has become ever larger and more complex; furthermore, because I'm human, my style has grown and evolved over the years, and the way I wrote things back in 1993 is almost invariably not the way I would do them now, so a lot of the code exists in a mixture of styles that makes maintenance very difficult.

By 2010, it had become absolutely clear to me that I could no longer reliably maintain the Pegasus Mail source code as it stood - it had become too large and had too much history and compromise in it... Add that to the fact that Mercury had also grown large and unwieldy, and I realized that I was going to have to find some way to reorganize and unify the programs - come up with common internal components so that I was maintaining less code, and the code I was having to maintain was more modern, more consistent, more streamlined.

At the time, I thought I could do this as a gradual process - replace a piece here, a piece there, and the programs would progressively converge on a new, maintainable standard... But I was wrong - or at best, naive. I would start working on modernizing one part of the program, only to find that if I were to do the work properly, I could salvage almost none of the old code - I would have to write it anew. Then I'd find that rewriting that part of the program couldn't proceed until I'd rewritten another part of the program on which it depended, so the process would start once more - and so on and on, over and over again.

Although it certainly wasn't my original intention, what I actually ended up doing was rewriting almost the entire program from scratch: this wasn't a process of modernization, as I had initially planned, but a process of writing a completely new program using an older one as a model.

It can be hard to get perspective on this process - on how large the work really is... But over time, I've come to look at what I do in terms of units I call "WaPs" - short for "War and Peace": Tolstoy's "War and Peace" (Oxford World Classics edition) has about 71,500 lines of text, so I defined 71,500 lines as 1 WaP, and use that to measure the size of the work I do. At the moment, the totals are roughly as follows:

Pegasus Mail 227,428 lines 3.181 WaP
Mercury 169,163 lines 2.366 WaP
Setup and Tools 154,321 lines 2.158 WaP
Manuals and Docs 96,210 lines 1.345 WaP
Totals: 647,122 lines 9.050 WaP

That means that the code I am working on is currently roughly the size of nine copies of War and Peace... That's a lot of lines. I further estimate that in the last four years, I have updated, added or removed around 200,000 of those lines of code (2.797 WaP), and although that's an estimate, it's on the low side if anything. The cold reality of these numbers is that the combined bulk of Pegasus Mail and Mercury is now beyond the ability of a single programmer to maintain and improve unless ways can be found to streamline the process as much as possible, and it's mainly this that I have been focusing on for the last few years. I have no intention of giving up on these programs that have been part of my life for so long, but I simply have to find ways of making them manageable.

So what have I actually done so far? Here's a potted summary, by no means complete:

Completely new MIME and RFC2822 message parsers (the MIME parser in particular is, I believe, as good as any in the entire industry).
Completely new RFC2822 message generator.
Completely new encryption support, including S/MIME support and support for encrypted databases, folders, contacts and lists.
Completely new contact manager (a kind of mega-addressbook that I personally think will be the most game-changing component in v5).
Totally overhauled user interface elements, including support for simple user-defined style changes (also known as "skinning").
Totally overhauled low-level tool sets - for example, a string library that can handle dynamic strings up to 1Gb or larger in size.
Completely new TCP/IP network layer with OpenSSL support (this component has already found its way into current versions of Pegasus Mail and Mercury).
Totally new console objects that allow the separation of the Mercury service processes from its user interface (this, in fact, was finished quite recently).
A totally new object-oriented framework to tie everything together.
A completely rewritten low-level message store.
The last two are so important that they deserve some elaboration: the object framework (which I call "OIF" internally) is the basis for all the new code, and will allow third-party add-ons to be first-class citizens, with the same access to the internals of Pegasus Mail and Mercury as the programs themselves: this desire to open up the "guts" of the programs in a consistent, effective manner that made it easy for people other than me to create powerful new features for them was a major goal of the redevelopment, and required enormous effort, although I believe the results are really good.

The new Message Store is an example of how difficult some of this stuff can end up being; I have been working on it for several years and in that time have ended up scrapping what I had done and restarting twice. The Message Store is the low-level code that implements the folders and messages you work with on a day-to-day basis, and I wanted something that provided a clear way forward to future capabilities without sacrificing compatibility with existing data; getting the right balance between past and future makes this process unusually difficult - had I been able to scrap everything and simply start from scratch, it would have been a far simpler process, but I value data compatibility above almost everything else: it is a matter of pride to me that a folder created by the DOS version of Pegasus Mail in 1990 is still readable by the latest Windows version in 2017, and will remain readable by v5, even though newer and vastly better folder formats will also be available.

Hindsight, as the old saying goes, has 20-20 vision, and in hindsight, I would have to say that I could have handled the process of updating and modernizing my programs considerably better than I have. Had I realized when I started what I was getting myself into, I would certainly have tried to find a different strategy, some way of making the process cleaner, more compartmentalized, and more inclusive of my user community... But in the end, the code would still have had to be written, and it was always going to take a long time no matter how I approached it.

I hope this rather over-wordy update (roughly 0.002 WaP in size) offers at least some explanation why the process of bringing out the next generation versions of Pegasus Mail and Mercury is taking such a long time: in offering it, I would also like to offer my thanks to my community of users, testers and supporters for their considerable patience and encouragement, and my hope that they will stick with me for just a little longer. I really do feel like there's light at the end now.

All my very best to you all,
Berenburger is offline   Reply With Quote

Old 13 May 2017, 06:51 PM   #2
evilquoll
Member
 
Join Date: May 2017
Location: Emergency temporary account of ROBERT.BAK
Posts: 36
It's to be hoped that PMail 5 arrives sometime soon; it's incompatible with Win10, and for obvious reasons running it in a sandboxed VM is not a workaround for that problem.
evilquoll is offline   Reply With Quote
Old 13 May 2017, 09:04 PM   #3
Jacinto
Essential Contributor
 
Join Date: Jun 2009
Posts: 395
To: Berenburger

Pegasus and Windows Commander are the only two Windows programs I have missed since I stopped doing Windows ten years ago.

I tried running them with Wine on Linux, but it wasn't the same.

Other than those two programs and with all due respect to Windows users, I haven't regretted my decision to leave Windows for Linux.

--
Jacinto
Jacinto is offline   Reply With Quote
Old 19 Jun 2018, 06:18 PM   #4
Berenburger
The "e" in e-mail
 
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,894
Building Blocks, June 2018

New post from David Harris about the Pegasus development.

Source: http://www.pmail.com/devnews.htm

Quote:
The release of WinPMail v4.73 gives me an opportunity to describe a little of the process that goes into developing a program like this - in particular, I'd like to provide a little insight into the new help system in v4.73, which is the first Pegasus Mail Version 5 feature to make it to release.

The new help system became a necessity when Microsoft retired the old WinHelp help format a few years ago, because they simply didn't replace it with anything suitable for developers like me who had an investment in it. The HTMLHelp program they provided was adequate for many developers, but suffered from a crucial flaw: because it depended on Internet Explorer for its HTML rendering, it could not load help files on shared volumes (IE won't do this, supposedly for security reasons). There's absolutely nothing worse than being stressed by a problem with a program, pulling up its help system, and seeing "Navigation Cancelled" instead of a help page, yet this is what happened if you tried to open an HTMLHelp help file that was installed on a server or NAS shared volume. There were mitigations for the problem, but they were poorly documented and technically demanding for anyone who wasn't a trained sysadmin.

So, after looking around to see what else was available and finding almost nothing, I began rolling my own using components I already had for Pegasus Mail. It was a long job - I estimate that developing the help compiler, the help display module, and the user interface for the help system took me about 400 hours over a period of years, and I know for a fact (having done it recently) that converting the old RTF source I used for the Pegasus Mail WinHelp help file to HTML for use in the new system took me 150 hours (yes, I was counting — it was a HORRIBLE job).

"But it's just a help system!" I can hear you saying; "it's just a little text and a window to display it in!". Well, the WinPMail help pages consist of more than 250 separate help files, all cross-indexed and linked, each formatted as consistently as possible: that's roughly 20,000 lines, or (for those of you who can recall my last developer update) about a third of a War-And-Peace, and that's just the basic text, without considering the markup, indexing and table of contents generation. The source code for the help compiler and the display module make up about another 15,000 lines of code - that's just the reality of working with a format as rich and complicated as HTML.

All of this just to replace something that Microsoft decided to remove because they couldn't be bothered keeping it up to date... That said, though, the result is pretty good, and I now have a help system that doesn't depend on anything outside my control, and that I can use in both Pegasus Mail and Mercury — I have only one piece of code to maintain, completely self-contained, unaffected by any changes to the programs that host it. And it is this that is really the goal of this update - to describe a process I am now applying to all my new development: code that stands on its own, can be used in either system, is available to extension developers, and requires minimal support in the long term.

This approach to developing things as building blocks gives flow-on benefits that can be guite interesting — so, as an example, the module I'm just finishing now that implements full S/MIME capability will allow Pegasus Mail to send and receive a full range of modern encrypted data, but will also allow Mercury to send out its list mail with digital signatures as proof of origin; in a similar fashion, the new Pegasus Mail Contact Manager (the replacement for the aging addressbook system) could easily serve as the basis for a Mercury vCard server — and there are countless other similar examples where having the one self-contained module will allow both programs to benefit: I'm expecting that there will even be crossovers I haven't anticipated yet.

Interesting times indeed.

All my very best to you all,

-- David --
Berenburger is offline   Reply With Quote
Old 23 Jul 2019, 07:33 AM   #5
Berenburger
The "e" in e-mail
 
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,894
The Eyes Have It

Somewhat worrying news about David Harris.

Source: http://www.pmail.com/devnews.htm

Quote:
In January this year, while I was taking a brief holiday with some friends in Christchurch (a city about 350km north of Dunedin, where I live), I ended up being admitted to hospital for emergency eye surgery on a detached retina. It was probably just as well that my friends are both doctors, since it meant that the condition was caught early.

The surgery was successful, but since that time I have had almost constant problems with my vision, partially as an after-effect of the operation, but mostly because of a rapidly-growing cataract that has formed on the lens of my left eye: for the last three months, I have had almost no useful vision in that eye. I am scheduled for priority cataract surgery to correct the problems, after which I should be back to normal (or even, they tell me, a little better than before the initial problem), but at this stage the best estimate the hospital can give me for when it will happen is "sometime in the next four months".

The effect this series of issues has had on my life, and on Pegasus Mail and Mercury, is quite considerable. My right eye has always been weaker than my left eye, but it now has to handle the entire burden of my vision. What this means for my user community is that I can only do screen work for about half an hour at a time before getting eye strain: I'm still working, but at a reduced rate, and my ability to do things like answer mail is heavily reduced.

Warning: raw emotional outpouring alert!
It's frustrating. So mind-bendingly frustrating! Some days it makes me so desperate I could almost cry.
End of emotional boilover

Now that I've got that off my chest, let me give you a small progress update. Mercury v5 is very near to going into beta - I just have to finish the mailing list manager overhaul that is required to support the new split interface and it will be ready. I do not expect a long beta period for v5 - I have done huge amounts of testing as I've worked on it (I had to - just about every source module in the program has changed significantly).

WinPMail has been lagging behind a bit as I put my efforts into Mercury, but I would rate progress as good, and believe I should be on target for rough public betas before the end of the year. WinPMail needs fewer structural changes than Mercury, but far more modernization, and there's much more of it.

I will put out another developer update soon laying out some of the changes you can expect to see in the new versions of both programs, but for now, I'm afraid, my half hour is up, and I'm squinting so hard at the screen I can feel my eyelids seizing up.

All the best!

-- David --
Berenburger is offline   Reply With Quote
Old 16 Feb 2020, 08:41 AM   #6
Berenburger
The "e" in e-mail
 
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,894
David Harris back on the track

Belated Christmas message.
Source: http://www.pmail.com/devnews.htm

Quote:
December 2019 - Several Issues

Ahh, December - the silly season. Turkey or ham, pavlova or apple pie, family and friends... Work should be the last thing on anyone's mind, but this is also a useful time to bring closure to the events of 2019 and look forward to the next year, so here we go...

In this instalment of the Developer News:

Eye update
Google, GMail and OAUTH2
OpenSSL updates
Other progress

Eye Update

After cataract surgery on November 26th I have made a steady recovery (hampered only by a minor but persistent infection which now appears to be under control) and now have good vision that will allow me to resume normal workloads in the New Year. Thank you to everyone who has sent me their kind wishes and support - I really appreciate it.

Google, GMail and OAUTH2

Many of you who use Google's GMail service may recently have received an ominously worded message from them advising that as of 2021, you will no longer be able to access their service using so-called "less secure applications". Instead, you will be required to use an authentication method called OAUTH2 in order to access your GMail account.

Now, this is a "hot button" issue for me - it will get me started, and if I'm not forcibly held down, it can be very hard to stop me from ranting almost indefinitely. I'll resist the urge to let loose, though, and simply say that (1) "Less-secure applications" is crude scare-mongering by Google that means little and is mainly intended to serve their political agenda, and that (2) OAUTH2 is among the most poorly-standardized and ill-considered hacks I've ever encountered in thirty years of serving the Internet.

That said, GMail's size makes it an effective bully, and regardless of what I might think about OAUTH2, as they say in Star Trek, "resistance is futile". Accordingly, we have been working on implementing support for OAUTH2 in Pegasus Mail, and already have a basic working proof-of-concept on which we can build going forward. We expect to have working OAUTH2 support for GMail (and at this stage, *only* GMail) well before it will be necessary for continued use of the service. One of the main flaws of OAUTH2 is that it has to be implemented on a service-by-service basis, and at this time we do not have plans to support services other than GMail, although we will review this position over time as required.

Our OAUTH2 support will also be available for Mercury users who use MercuryD to access GMail accounts, but it may take a little more effort to use. We will publish more about this as work proceeds.

OpenSSL Updates

We will shortly be releasing an update for both Pegasus Mail and Mercury that implements a more modern version of the OpenSSL libraries used to provide secure access to remote servers and services. The update will include a version of OpenSSL called 1.0.2t, which is already right near the end of its support life, but which should be fine in the short term. The reason we are not issuing support for the current 1.1.1 build of OpenSSL is because we simply have not been able to produce a build that will pass its own internal self-tests. Once we've got this issue sorted out, there will be another update that includes the more up-to-date version of OpenSSL. The interim release will be suitable for use in Pegasus Mail v4.73 and later, and Mercury v4.7 and v4.8.

Other progress

After a hiatus because of my eye problems, work has recommenced on Mercury v5 and Pegasus Mail v5. Mercury will be the first to be released, mostly because it is smaller and has less user-interface than Pegasus Mail. I hope to have builds of Mercury v5 available to testers and interested users in the first three months of the New Year, while Pegasus Mail will probably be slightly further out. I appreciate everyone's ongoing patience with the seemingly neverending delays in these releases - I can only promise you that there is progress, and that I'm totally committed to getting these new versions released.

------------------

All my very best to everyone for the Christmas season, and my hopes for a better New Year for all.
Berenburger is offline   Reply With Quote
Old 25 Jul 2020, 09:34 AM   #7
Berenburger
The "e" in e-mail
 
Join Date: Sep 2004
Location: The Netherlands
Posts: 2,894
Community Site Problems

http://www.pmail.com/comprob.htm

Quote:
The Pegasus Mail Community site has been taken offline while we redevelop
it into a more modern format. The domain name will remain the same,
and we hope to have a new community in operation as soon as possible.

Please watch this space for further developments.
Berenburger is offline   Reply With Quote
Old 19 May 2022, 03:39 AM   #8
FredOnline
The "e" in e-mail
 
Join Date: Apr 2011
Location: Manchester UK
Posts: 2,616
Pegasus Mail Newsflash

http://www.pmail.com/newsflash.htm

Quote:
So, having spent hundreds of frustrating hours developing a working OAUTH2 solution for GMail, I am defeated at the final hurdle. Google's demands mean that I am simply not going to be able to support GMail past May 31st, however much it hurts me to feel that I am letting my users down.
FredOnline is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT +9. The time now is 04:58 PM.

 

Copyright EmailDiscussions.com 1998-2022. All Rights Reserved. Privacy Policy