Weird sender name text

musicmouse · 3 Feb 2023, 02:32 AM

I have a webshop installation where emails show a weird sender. The sender name should be some text in Hebrew but instead I get a long MIME string that starts with "=?utf-8?Q?=D7=A2=D7=95=D7=9C=D7=9D_=D7=94=D7=9E=D7=95?= =?utf-8?Q?" at the place where the name should be.

I have two nearly identical installations - one is an upgrade of the other - that use the same account on the same SMTP server. Yet one works ok and the other not.

Tow things are remarkable:

- When I look at the source of the mail of the erroneous installation I see a X-Google-Original-From: entry that contains the correct sender name in Hebrew followed by my (the receiver!) email address. In the correct installation that field is missing.

- This problem happens only on the pc. On the mobile phone I see the sender name correctly.

What could cause these problems?

JeremyNicoll · 4 Feb 2023, 02:50 AM

I presume that by "webshop installation" you mean some sort of e-commerce package installed on a webserver?

If so, are both installations on the same server? Does either (or indeed both) installation override defaults system software on that server?

If not, do the two servers have the exact same versions of other software (eg php, and anything else that they use)?

Do the installations have configuration options, and are all those choices the same?

I'm guessing that the value that's encoded as: "=?utf-8?Q?=D7=A2=D7=95=D7=9C=D7=9D_=D7=94=D7=9E=D7=95?= =?utf-8?Q?" is correct. That (or at least something of that format) is what I would expect to see in any header where a Hebrew or Arabic or Cyrillic or Chinese or ... <whatever> name was placed - any name that isn't formed with plain ascii characters.

JUST TO BE CLEAR - that value is what I would expect to be in the headers of both the mails you see displayed properly, and the mails that look weird to you. The problem is not that that value is there; it's that some system is not displaying the decoded value.

So that makes me wonder where are you seeing the problem, which you describe as "on the pc"? How do you look at email "on the pc"? Are you using a webmail system (ie using a browser to login to someone's website to look at the emails), or a standalone email client (eg Thunderbird)?

Maybe there's an option in the webmail system or email client that dictates whether encoded headers are displayed in their raw (ie encoded) format, or their decoded value is shown. Some webmail systems/clients maybe aren't versatile enough.

musicmouse · 4 Feb 2023, 03:32 AM

I am seeing this when I receive this in Gmail. And I don't think that that is normal.

When I put the string in a MIME decoder (https://dogmamix.com/MimeHeadersDecoder/) I get valid output. But what use is that when everyone who receives mail from this installation doesn't see it?

In mails from the older version of the software I see the Hebrew characters in the sender and the =D7= garbage is nowhere to be found.

JeremyNicoll · 4 Feb 2023, 05:08 AM

Quote:

Originally Posted by musicmouse

In mails from the older version of the software I see the Hebrew characters in the sender and the =D7= garbage is nowhere to be found.

You are misunderstanding. The D7 (etc) stuff is NOT garbage. It's how email works.

If Gmail (or your phone) allow you to look at the "raw email" or "email source" ie the whole set of headers and text that is inside any of your received emails (not just those from the e-commerce application), you will see that Hebrew / Chinese / Cyrillic characters etc in headers are always represented by UTF-encoding (or possibly in some case some other form of encoding).

It's because headers can only contain ordinary ASCII characters. That's the way it works.

Your problem is to find out whether the e-commerce application is sending malformed emails (but that's unlikely if they display ok on your phone, unless your phone's using email software that's also broken), or whether there's some way to configure Gmail to work differently. Perhaps you should also be asking the authors of the e-commerce package or looking for forums that support that software?

The fact that you see a "X-Google-Original-From" header suggests there's a Google process involved, though headers with names starting "X" are (in RFC terms) unofficial - they could be added anywhere with any non-standard meaning.

musicmouse · 5 Feb 2023, 03:15 AM

Until now I had used "show original" in Gmail. Now I chose to download the email as an eml file and to have a look at that. That delivered some interesting result.

The subject of the mail, that in "show original" is show in Hebrew letters is shown there in the familiar =D7= MIME encoding.

But the From: part seems double encoded. It starts like this:

From: =?UTF-8?B?PT91dGYtOD9RPz1ENz1BMj1ENz05NT1ENz05Qz1ENz05RF89

How should I interpret this?

BTW: Mozilla Thunderbird doesn't display the sender name correctly either.

JeremyNicoll · 5 Feb 2023, 07:04 AM

In the earlier example, which started =?utf-8?Q? then Q meant something called "quoted printable" which is one way to encode characters.

In this case you have ?UTF-8?B? where the "B" means that what follows is encoded as "Base64".

If I cut & paste "PT91dGYtOD9RPz1ENz1BMj1ENz05NT1ENz05Qz1ENz05RF89" into the first box at: https://www.base64decode.org/ and click "decode" it shows it's equivalent to

=?utf-8?Q?=D7=A2=D7=95=D7=9C=D7=9D_=

I've no idea why some sending software would do that. You'd need to ask the programmers who wrote the code that generates the emails.

musicmouse · 7 Feb 2023, 08:38 PM

That doesn't sound good.

I tried to fix the code but found it too hard to understand (150 php files and probably over a thousand functions). Does anyone here know Swiftmailer and know where it might add this Base64 encoding? I could find where it does the quoted printable encoding but not the later stage where it does the base64 encoding. There are some base64 files but they don't seem to be called for this purpose.

Unfortunately the code is no longer supported.

somdcomputerguy · 7 Feb 2023, 09:27 PM

A feature-rich PHP Mailer – Documentation – Swift Mailer https://swiftmailer.symfony.com/docs/introduction.html

Quote:

Swift Mailer will stop being maintained at the end of November 2021.

Please, move to Symfony Mailer at your earliest convenience. Symfony Mailer is the next evolution of Swift Mailer. It provides the same features with support for modern PHP code and support for third-party providers.

- Bruce

musicmouse · 8 Feb 2023, 07:23 PM

I am starting to wonder whether it is really Swiftmailer that causes this problem.

What I now see is two lines in the header:
From: encrypted name <office@sender.com>
X-Google-Original-From: real name <myaccount@gmail.com>

In the version that works there is no X-Google-Original-From:. Instead I just see what is expected:
From: real name <office@sender.com>

So what is this X-Google-Original-From: doing and where is that done? And why?

JeremyNicoll · 9 Feb 2023, 03:23 AM

You mentioned right back at the start that "This problem happens only on the pc. On the mobile phone I see the sender name correctly".

What you didn't say (unless I missed it) is

- whether when you compare the pc & the phone, you're looking at the exact same email (eg on gmail servers) or two different emails sent to two different email servers (eg gmail and somewhere else)

- using the same software (eg the same browser with the same settings), if both places the mails are sent to are accessed by webmail

I think it would be interesting to have the two different e-commerce applications send emails to somewhere, anywhere, that is not Gmail. I can't recommend anywhere (as I only have paid accounts) ... but maybe someone could recommend somewhere.

Also, do the e-commerce applications store "log copies" of outbound emails (as normal email client usually do)? If the problem lies in what's being sent, you need to look at those, as well as what arrives.

SideshowBob · 9 Feb 2023, 05:10 AM

Where the header was not decoded, did the email have a header like:

Mime-Version: 1.0

(it may have an optional string in brackets). If this header is missing the email should not be treated as mime.

base64 encoded headers are quite common because they are more efficient unless the text is mostly ascii. Some software always uses it rather than going to the trouble of making a decision per email.

hadaso · 9 Feb 2023, 08:18 PM

It looks as if the problematic email is being recent, and the software that resends it reencodes some headers, but doesn't decode the header before reencoding (so a header that starts as qp encoded gets encoded in base64, resulting in double encoding, instead of being first decoded into utf-8 plain text and then encoded using base64).
To really understand what's happening all the headers are needed (to see how the two versions of the email have been routed to the destination).

3 Feb 2023, 02:32 AM	#1
musicmouse Junior Member Join Date: Feb 2023 Posts: 5	Weird sender name text I have a webshop installation where emails show a weird sender. The sender name should be some text in Hebrew but instead I get a long MIME string that starts with "=?utf-8?Q?=D7=A2=D7=95=D7=9C=D7=9D_=D7=94=D7=9E=D7=95?= =?utf-8?Q?" at the place where the name should be. I have two nearly identical installations - one is an upgrade of the other - that use the same account on the same SMTP server. Yet one works ok and the other not. Tow things are remarkable: - When I look at the source of the mail of the erroneous installation I see a X-Google-Original-From: entry that contains the correct sender name in Hebrew followed by my (the receiver!) email address. In the correct installation that field is missing. - This problem happens only on the pc. On the mobile phone I see the sender name correctly. What could cause these problems?

4 Feb 2023, 02:50 AM	#2
JeremyNicoll Essential Contributor Join Date: Dec 2017 Location: Scotland Posts: 492	I presume that by "webshop installation" you mean some sort of e-commerce package installed on a webserver? If so, are both installations on the same server? Does either (or indeed both) installation override defaults system software on that server? If not, do the two servers have the exact same versions of other software (eg php, and anything else that they use)? Do the installations have configuration options, and are all those choices the same? I'm guessing that the value that's encoded as: "=?utf-8?Q?=D7=A2=D7=95=D7=9C=D7=9D_=D7=94=D7=9E=D7=95?= =?utf-8?Q?" is correct. That (or at least something of that format) is what I would expect to see in any header where a Hebrew or Arabic or Cyrillic or Chinese or ... <whatever> name was placed - any name that isn't formed with plain ascii characters. JUST TO BE CLEAR - that value is what I would expect to be in the headers of both the mails you see displayed properly, and the mails that look weird to you. The problem is not that that value is there; it's that some system is not displaying the decoded value. So that makes me wonder where are you seeing the problem, which you describe as "on the pc"? How do you look at email "on the pc"? Are you using a webmail system (ie using a browser to login to someone's website to look at the emails), or a standalone email client (eg Thunderbird)? Maybe there's an option in the webmail system or email client that dictates whether encoded headers are displayed in their raw (ie encoded) format, or their decoded value is shown. Some webmail systems/clients maybe aren't versatile enough. Last edited by JeremyNicoll : 4 Feb 2023 at 02:52 AM. Reason: added JUST TO BE CLEAR part

4 Feb 2023, 03:32 AM	#3
musicmouse Junior Member Join Date: Feb 2023 Posts: 5	I am seeing this when I receive this in Gmail. And I don't think that that is normal. When I put the string in a MIME decoder (https://dogmamix.com/MimeHeadersDecoder/) I get valid output. But what use is that when everyone who receives mail from this installation doesn't see it? In mails from the older version of the software I see the Hebrew characters in the sender and the =D7= garbage is nowhere to be found. Last edited by musicmouse : 4 Feb 2023 at 03:52 AM.

5 Feb 2023, 03:15 AM	#5
musicmouse Junior Member Join Date: Feb 2023 Posts: 5	Until now I had used "show original" in Gmail. Now I chose to download the email as an eml file and to have a look at that. That delivered some interesting result. The subject of the mail, that in "show original" is show in Hebrew letters is shown there in the familiar =D7= MIME encoding. But the From: part seems double encoded. It starts like this: From: =?UTF-8?B?PT91dGYtOD9RPz1ENz1BMj1ENz05NT1ENz05Qz1ENz05RF89 How should I interpret this? BTW: Mozilla Thunderbird doesn't display the sender name correctly either.

5 Feb 2023, 07:04 AM	#6
JeremyNicoll Essential Contributor Join Date: Dec 2017 Location: Scotland Posts: 492	In the earlier example, which started =?utf-8?Q? then Q meant something called "quoted printable" which is one way to encode characters. In this case you have ?UTF-8?B? where the "B" means that what follows is encoded as "Base64". If I cut & paste "PT91dGYtOD9RPz1ENz1BMj1ENz05NT1ENz05Qz1ENz05RF89" into the first box at: https://www.base64decode.org/ and click "decode" it shows it's equivalent to =?utf-8?Q?=D7=A2=D7=95=D7=9C=D7=9D_= I've no idea why some sending software would do that. You'd need to ask the programmers who wrote the code that generates the emails.

7 Feb 2023, 08:38 PM	#7
musicmouse Junior Member Join Date: Feb 2023 Posts: 5	That doesn't sound good. I tried to fix the code but found it too hard to understand (150 php files and probably over a thousand functions). Does anyone here know Swiftmailer and know where it might add this Base64 encoding? I could find where it does the quoted printable encoding but not the later stage where it does the base64 encoding. There are some base64 files but they don't seem to be called for this purpose. Unfortunately the code is no longer supported.

8 Feb 2023, 07:23 PM	#9
musicmouse Junior Member Join Date: Feb 2023 Posts: 5	I am starting to wonder whether it is really Swiftmailer that causes this problem. What I now see is two lines in the header: From: encrypted name <office@sender.com> X-Google-Original-From: real name <myaccount@gmail.com> In the version that works there is no X-Google-Original-From:. Instead I just see what is expected: From: real name <office@sender.com> So what is this X-Google-Original-From: doing and where is that done? And why?

9 Feb 2023, 03:23 AM	#10
JeremyNicoll Essential Contributor Join Date: Dec 2017 Location: Scotland Posts: 492	You mentioned right back at the start that "This problem happens only on the pc. On the mobile phone I see the sender name correctly". What you didn't say (unless I missed it) is - whether when you compare the pc & the phone, you're looking at the exact same email (eg on gmail servers) or two different emails sent to two different email servers (eg gmail and somewhere else) - using the same software (eg the same browser with the same settings), if both places the mails are sent to are accessed by webmail I think it would be interesting to have the two different e-commerce applications send emails to somewhere, anywhere, that is not Gmail. I can't recommend anywhere (as I only have paid accounts) ... but maybe someone could recommend somewhere. Also, do the e-commerce applications store "log copies" of outbound emails (as normal email client usually do)? If the problem lies in what's being sent, you need to look at those, as well as what arrives.

9 Feb 2023, 05:10 AM	#11
SideshowBob Essential Contributor Join Date: Jan 2017 Posts: 278	Where the header was not decoded, did the email have a header like: Mime-Version: 1.0 (it may have an optional string in brackets). If this header is missing the email should not be treated as mime. base64 encoded headers are quite common because they are more efficient unless the text is mostly ascii. Some software always uses it rather than going to the trouble of making a decision per email.

9 Feb 2023, 08:18 PM	#12
hadaso The "e" in e-mail Join Date: Oct 2002 Location: Holon, Israel. Posts: 4,857	It looks as if the problematic email is being recent, and the software that resends it reencodes some headers, but doesn't decode the header before reencoding (so a header that starts as qp encoded gets encoded in base64, resulting in double encoding, instead of being first decoded into utf-8 plain text and then encoded using base64). To really understand what's happening all the headers are needed (to see how the two versions of the email have been routed to the destination).