Weird Undeliverable Mail messages

[chrisjj]

I have a Fastmail a rule on incoming mail to send a copy to Gmail.

I got a spate of "Undelivered Mail Returned to Sender" messages that make no sense.

The Diagnostic code is:

<[snip]@gmail.com>: host alt4.gmail-smtp-in.l.google.com[142.250.153.27]
said: 421-4.7.27 Your email has been rate limited because SPF
authentication didn't 421-4.7.27 pass for this message. Gmail requires all
bulk email senders to 421-4.7.27 authenticate with SPF. 421-4.7.27
421-4.7.27 Authentication results: 421-4.7.27 SPF
[srs.messagingengine.com] with ip: [202.12.124.199] = did not 421-4.7.27
pass 421-4.7.27 421-4.7.27 To set up SPF for your sending domains, visit
421-4.7.27 https://support.google.com/a?p=setup-spf 421-4.7.27 To learn
more about Gmail requirements for bulk senders, visit 421 4.7.27
https://support.google.com/a?p=sender-guidelines.
4fb4d7f45d1cf-5cbb63661c6si7634801a12.665 - gsmtp (in reply to end of DATA
command)

which I take to mean Gmail is rejecting the Fastmail Send a copy attempt. Fine.

But:

1 They are To: SRS0=k9hx=RZ=fastmail.fm=chrisjj@srs.messagingengine.com . Er, why? Some proxy of mine, trying to do the send a copy?

2 They say "I'm sorry to have to inform you that your message could not be delivered" and enclose a message which is NOT mine. It is from X to me. It is not being returned to the sender. It is being send to me, the FM recipient. Has FM mistaken this message for mine because it was my account that was trying to do the send a copy?

3 One is followed by another that is identical except the diagnostic code differs in just one line

4fb4d7f45d1cf-5cbb6307cf4si7633028a12.248 - gsmtp (in reply to end of DATA

Why this second instance?

4 Despite that the code 421 means "Your message was temporarily deferred by the recipient server." the Subject is: "Undelivered Mail Returned to Sender" and the report makes no suggestion that delivery will be retried.

[hadaso]

I don't really understand the SRS mechanism. Anyway I guess the second non-delivery message is probably in response to a second attempt to deliver, since the SMTP error code was 421 which means a temporary failure to deliver.
If the message was sent by you using "Send a copy" then I guess Fastmail correctly directs the non-delivery message to you (the one that initiated the delivery of the message) and not to the original sender of the message (that has successfully sent it to you).

[chrisjj]

Originally Posted by hadaso

Anyway I guess the second non-delivery message is probably in response to a second attempt to deliver,

Yes it is identical in time.

Originally Posted by hadaso

since the SMTP error code was 421 which means a temporary failure to deliver.

Good point. I've added that contradiction to the question. EDIT: FM confirms this is its interpretation:

"If the code starts with 4 (also referred to as a 4xx code), this means that the message has been soft bounced. A soft bounce occurs if the mail system believes that the issue causing the bounce may be temporary, and decides to continue making attempts to deliver the message for a short amount of time (often 1 day or so). If the issue is resolved within that time period, the message will be delivered. If the issue is not resolved, you will then receive a hard bounce message with a 5xx code." https://www.fastmail.help/hc/en-us/a...es-bounce-back

Originally Posted by hadaso

If the message was sent by you using "Send a copy"

How can I tell from this report?

Originally Posted by hadaso

then I guess Fastmail correctly directs the non-delivery message to you (the one that initiated the delivery of the message) and not to the original sender of the message (that has successfully sent it to you).

OK, but is not returning to my regular address.

[chrisjj]

PS Astonishingly FM support says that upon sending this "Undelivered Mail Returned to Sender" message with soft-bounce code 421, FM gave up trying to send the message. I await their clarification on what has made their server mistake this soft bounce for a hard one.

[hadaso]

The text that accompanies the 421 code says that the message was not delivered because of failing SPF authentication, and it suggests the sender setup SPF for the domain. So it is a temporary failure only if the sender changes SPF settings and retries delivery, but if the sender does not change the SPF settings there is no reason to reattempt delivery because it would fail again for the same reason.

[chrisjj]

Originally Posted by hadaso

The text that accompanies the 421 code says that the message was not delivered because of failing SPF authentication

Actually it does not say the message was not delivered. It says "rate limited".

Originally Posted by hadaso

So it is a temporary failure only if the sender changes SPF settings and retries delivery, but if the sender does not change the SPF settings there is no reason to reattempt delivery because it would fail again for the same reason.

Regardless, that's not FM's interpretation. FM clearly states 4XX means "decides to continue making attempts to deliver the message".

And then "If the issue is not resolved, you will then receive a hard bounce message with a 5xx code." I did not receive one.

Hence this looks to me like a FM fail to operate as advertised.

[hadaso]

Originally Posted by chrisjj

Regardless, that's not FM's interpretation. FM clearly states 4XX means "decides to continue making attempts to deliver the message".

It's not FM's interpretation. It is part of the specification of the SMTP.

And then "If the issue is not resolved, you will then receive a hard bounce message with a 5xx code." I did not receive one.

I think this is a mistake: e.g. if the recipient is over quota, each delivery attempt would produce exactly the same 4xx error code, and never a 5xx code. A bounce message would be produced by FM when their system decides to stop trying to deliver the message, but the code from the recipient's server would still be a 4xx code.

[SideshowBob]

This is most likely a temporary problem with DNS - SPF for SRS forwarded mail isn't one of the things that FM deliberately break.

[chrisjj]

Originally Posted by hadaso

It is part of the specification of the SMTP.

If your "it" is "4XX means "decides to continue making attempts to deliver the message".", then I agree.

Originally Posted by hadaso

I think this is a mistake:

I take "it" to be FM's "If the issue is not resolved, you will then receive a hard bounce message with a 5xx code."

I see nothing suggesting this is a mistake. I see only that that this documentation and the behaviour fail to accord. I would bet the behaviour did originally accord, but FM has downgraded it and neglected to sync the documentation.

Originally Posted by hadaso

e.g. if the recipient is over quota, each delivery attempt would produce exactly the same 4xx error code, and never a 5xx code.

I disagree. Over quota state might be rectified before the next delivery attempt. Just as the current case's /rate limit/ might allow the next delivery attempt.

Originally Posted by hadaso

A bounce message would be produced by FM when their system decides to stop trying to deliver the message, but the code from the recipient's server would still be a 4xx code

That's the current behaviour, but it fails to accord with the docs, which say 4xx means still trying and only 5xx means permanent reject.

My complaint is that after sender received a 4xx (soft bounce), neither delivery nor 5xx (hard bounce) occurred. That's a serious failure of chain of custody.

[chrisjj]

Originally Posted by SideshowBob

This is most likely a temporary problem with DNS

Temporary? Four days here. On this the fourth day, I have received two more bounce messages exhibiting #1,#2 and #4.

Originally Posted by SideshowBob

- SPF for SRS forwarded mail isn't one of the things that FM deliberately break.

But accidental breakage is a thing

[hadaso]

Originally Posted by chrisjj

My complaint is that after sender received a 4xx (soft bounce), neither delivery nor 5xx (hard bounce) occurred. That's a serious failure of chain of custody.

The 4xx or 5xx code is produced by the recipient's server, not by FM. If the recipient's system keeps producing the same 4xx rejection then eventually FM would give up and not retry indefinitely to deliver the message. And then it would produce a non-delivery report, and it would include whatever the response of the recipient's was. What's wrong with Fastmail's documentation in this respect is that they cannot promise a 5xx code since they are not the one that produces the code. instead they should state that when they receive a 5xx code they stop trying to deliver and send a non-delivery report, and when the get a 4xx code they keep trying to deliver for x days and if unsuccessful after x days (or if the code changes to 5xx) they stop trying and send a non-delivery report.

[chrisjj]

Originally Posted by hadaso

The 4xx or 5xx code is produced by the recipient's server, not by FM.

Understood. But the definition of the codes' meaning to the FM-using recipient is produced by FM. As is the bounce message. I see no problem with the recipient's server's code. Just with FM's response to it.

Originally Posted by hadaso

If the recipient's system keeps producing the same 4xx rejection then eventually FM would give up and not retry indefinitely to deliver the message. And then it would produce a non-delivery report,

No problem there.

Originally Posted by hadaso

and it would include whatever the response of the recipient's was.

Problem there. If this response was 4xx, then by FMs definition that report would be a soft not hard bounce.

Originally Posted by hadaso

What's wrong with Fastmail's documentation in this respect is that they cannot promise a 5xx code since they are not the one that produces the code.

FM has promised a 5xx code so I would say what is wrong is FM is failing to make good on its promise.

Originally Posted by hadaso

instead they should state that when they receive a 5xx code they stop trying to deliver and send a non-delivery report, and when the get a 4xx code they keep trying to deliver for x days and if unsuccessful after x days (or if the code changes to 5xx) they stop trying and send a non-delivery report.

I personally would certainly not assume that's the current behaviour. I've received "Warning: could not send message for past 4 hours" in the past, but not even once during this current spate of trouble, despite that frequently the delay has been over 24hrs.

And even if it was the current behaviour, that statement is unsafe until there's an amended definition that lets the user detect a "non-delivery report" i.e. hard bounce.