Yeah, I was actually thinking of something similar to protonmail, where the mail is encrypted on their side. I want it to be so secure that if I lose the password, the mail can't be recovered.
On the other hand, I'm very satisfied with Fastmail. There's nothing like it out there. I used some time tonight to check out the real secure ones, but while secure, they were severely lacking.
For reference, I checked out these: Hushmail, Protonmail, CounterMail, Tutanota, Posteo and Mailfence. They may be secure, but they do not have the features I need. Protonmail came closest, but it would cost € 30.00 / month to get a marginally usable feature set.
So I'll stick with Fastmail for now.