EmailDiscussions.com  

Go Back   EmailDiscussions.com > Email Service Provider-specific Forums > FastMail Forum
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
Stay in touch wirelessly

FastMail Forum All posts relating to FastMail.FM should go here: suggestions, comments, requests for help, complaints, technical issues etc.

Reply
 
Thread Tools
Old 28 Jan 2022, 07:01 AM   #1
camner
Cornerstone of the Community
 
Join Date: Jul 2002
Location: Tacoma, WA
Posts: 642
Custom Sieve code suddenly fails when worked before

I have some custom Sieve code that has not been changed in over a year. Suddenly, a couple of days ago, SOME (but not all) of the rules started to fail, with mail ending up in my Inbox rather than in another folder in accordance with the sieve rules.

There is consistency in that the rules that fail always fail, and the rules that don't fail, never fail.

FM's Sieve Tester reports that the failing rules should result in the mails being routed as I intend.

I'm not sure how to troubleshoot this, because it's only SOME rules that now consistently fail.
camner is offline   Reply With Quote

Old 28 Jan 2022, 09:57 AM   #2
JeremyNicoll
Essential Contributor
 
Join Date: Dec 2017
Location: Scotland
Posts: 483
Are you quite certain that nothing's changed in the emails that are no longer being processed the way you expect? For example, the sender (if it's an organisation) might have changed their email infrastructure.


I think you'd have to ask Fastmail if they've changed anything Sieve-related. For example, maybe they've installed a newer version of the Sieve engine.

Apart from that, your Sieve code is presumably executed before the FM code, which would imply that none of your logic is matching anything. It might be useful temporarily to append to your code a

if something_that_is_always_true do addheader something

just to show that that is happening, and perhaps (if that does happen) then sprinkle some addheaders into earlier code.

I assume that there's a Sieve log (presumably on one of FM's servers). If there is, it's a pity that any errors in Sieve processing aren't somehow accessible to users. Maybe we should ask for that to be added?


RFC 5228 - https://datatracker.ietf.org/doc/html/rfc5228 - says in section 2.10.8 that:

When an error occurs in a Sieve script, all processing stops.

(which presumably means that mail goes to INBOX as it would have done if there'd been no Sieve logic at all). It also says

When an error happens, implementations MUST notify the user that an
error occurred and which actions (if any) were taken, and do an
implicit keep.

and it seems to me that while the "keep" happens, the notification does not.

Last edited by JeremyNicoll : 28 Jan 2022 at 10:13 AM.
JeremyNicoll is offline   Reply With Quote
Old 28 Jan 2022, 07:18 PM   #3
xyzzy
Essential Contributor
 
Join Date: May 2018
Posts: 474
I was going to suggest the following:

Have you tried running your script through Sieve Tester? Copy your entire script and the email message's raw text over to Sieve Tester and of if doesn't hit any of the mailboxidexists commands you should be able reproduce the behavior. And as noted above, if it doesn't produce the expected behavior, sprinkle in a few addheader traces for debugging.

Note, if you are familiar with the Sieve script's "Copy to clipboard" button note that for some reason "they" moved it from the top of the Sieve window to the bottom. Why? I don't know. I initially thought they had removed it too along with narrowing the code display.

Last edited by xyzzy : 28 Jan 2022 at 07:45 PM.
xyzzy is offline   Reply With Quote
Old 29 Jan 2022, 12:50 AM   #4
hadaso
The "e" in e-mail
 
Join Date: Oct 2002
Location: Holon, Israel.
Posts: 4,801
Quote:
Originally Posted by JeremyNicoll View Post
...
When an error happens, implementations MUST notify the user that an
error occurred and which actions (if any) were taken, and do an
implicit keep.

and it seems to me that while the "keep" happens, the notification does not.
This depends on how the term "user" is interpreted, and also on how the term "notify" is interpreted.
For instance "user" can be interpreted as a FastMail process that sends messages to the Sieve engine and receives results and acts accordingly, and "notify" can mean a line written to a log accessible to this "user".
Notification might mean that a header is written into the message that records Sieve failure (and if such a header is written then al these messages could be found by searching the inbox).



I think a detailed Sieve log can be useful to me because I could probably erase hundreds of lines from my script that have not been used for years, but not knowing which rules are still active sometimes and which are not I tend to just leave them there as long as they are not breaking anything..
hadaso is offline   Reply With Quote
Old 29 Jan 2022, 01:57 AM   #5
camner
Cornerstone of the Community
 
Join Date: Jul 2002
Location: Tacoma, WA
Posts: 642
Thanks for your replies. I’ll dig into this more deeply and see if the sender of the emails that fail has changed something
camner is offline   Reply With Quote
Old 31 Jan 2022, 02:11 AM   #6
camner
Cornerstone of the Community
 
Join Date: Jul 2002
Location: Tacoma, WA
Posts: 642
Well, color me embarrassed…I’m almost ashamed to actually admit to what happened!

Although I had NOT changed my sieve script, I DID move some folders, and that made the sieve rules fail (duh!).

Question: If one uses the FM interface to create rules, rather than Sieve, will the rules automatically update if the folders are moved, renamed, etc., or will they fail, too? If they update themselves to take into account changes such as folder paths and names, than this would lead one to consider changing one’s sieve rules to FM interface rules. I’ve been avoiding that because I have so many custom rules (mostly for email sorting) and didn’t want to take the time to create the new rules, but maybe I should reconsider.
camner is offline   Reply With Quote
Old 31 Jan 2022, 05:54 AM   #7
JeremyNicoll
Essential Contributor
 
Join Date: Dec 2017
Location: Scotland
Posts: 483
Quote:
Originally Posted by camner View Post
Question: If one uses the FM interface to create rules, rather than Sieve, will the rules automatically update if the folders are moved, renamed, etc
They certainly do update for folder renames.

(Whereas eg RoundCube filters do not, at least on the version of RC - v1.4.13 - I use elsewhere.)
JeremyNicoll is offline   Reply With Quote
Old 1 Feb 2022, 06:09 AM   #8
camner
Cornerstone of the Community
 
Join Date: Jul 2002
Location: Tacoma, WA
Posts: 642
I took a look at the code generated when one creates a rule using the FM interface.

Here's an excerpt from the code generated for a rule that moves to a specific folder an email that meets the conditions of the rule:
Code:
if mailboxidexists "c5c83b43-921a-4a80-8324-e36b774e6b68" {
    set "L3_VITA_RAC_Volunteer_Hours" "Y";
    set "skipinbox" "Y";
  }
It looks as if what is happening here is that FM is looking to see if a specific folder exists (not by name) and then moves it there. Interestingly, it seems just to set a flag, and not use the "move" command (which for all I know, just sets a flag!).

I don't know what "L3" means, but the "VITA_RAC_Volunteer_Hours" is a path. The folder "RAC_Volunteer Hours" is at Inbox.VITA.RAC_Volunteer Hours.

So, it may well be worth the effort to change some of my Sieve rules to FM interface rules. The latter seems more robust.
camner is offline   Reply With Quote
Old 1 Feb 2022, 07:53 AM   #9
JeremyNicoll
Essential Contributor
 
Join Date: Dec 2017
Location: Scotland
Posts: 483
Quote:
Originally Posted by camner View Post
It looks as if what is happening here is that FM is looking to see if a specific folder exists (not by name) and then moves it there. Interestingly, it seems just to set a flag, and not use the "move" command (which for all I know, just sets a flag!).
What actually happens is that they have all that logic, like you spotted, which sets flags.

Further on, you'll find more logic that tests the named flags, and then moves the mail to the required folder (at least that's what I saw when I made some notes about it in March 2021).

I've not worked out what the Lnnn numbers mean, precisely, but they might just be the nn'th folder you defined, even if it didn't always have the name it has now. For eg L123_xxxx might be the 123rd folder you defined. Or it might mean that status info about that folder is stored in (say) the 123rd element of an array; remember that they've got to have a quick way to work out things like the colour to draw a folder name in, in their GUI. Or it might have something to do with the interaction between labels and folders.
JeremyNicoll is offline   Reply With Quote
Old 1 Feb 2022, 03:16 PM   #10
xyzzy
Essential Contributor
 
Join Date: May 2018
Posts: 474
The Lx_ prefixes are just used to name the switches to guide section 4d's (Do rule actions) behavior. The x numbers start at 0 and increased by 1 for each new rule generated based on the UI instructions. When FM switch a new UI (about a year ago?) they went crazy with switches!

I'm not entirely clear on the use of those mailboxid's but if I reall it has something to do with some IMAP edge condition where a user could rename an folder and the use of the mailboxid allows the use of the original folder no mater what it gets renamed to. But don't hold me to that!

Caution: if you run your script through Sieve Tester don't let it "see" one of those mailboxidexists (or mailboxexists if you happen to use that in some of your own Sieve code). Sieve Tester will just abort without producing any output.

Last edited by xyzzy : 1 Feb 2022 at 03:29 PM.
xyzzy is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Forum Jump


All times are GMT +9. The time now is 06:07 PM.

 

Copyright EmailDiscussions.com 1998-2022. All Rights Reserved. Privacy Policy