core-8.6.80.621: German Umlauts broken in the subject line of sent mails

Gerald

Using todays core-8.6.80.621 (also 8.6.80.590 from Monday), Umlauts are broken in the subject line of sent mails.
I initially thought someone broke Outlook 2010 support unintentionally, but this is also present without using Outlook 2010.

For Example:

Mail sent from external Kopano (8.6.80.621) Outlook 2010 to me (different Kopano 8.6.80.621 system): broken in IMAP (Thunderbird) and Outlook 2016
Mail sent from my own Outlook 2016 to myself: ok in Outlook 2016, but not ok in WebApp and IMAP (Thunderbird)
Mail sent from Outlook 2010 to myself (Outlook 2010): broken

Here are two screenshots from Thunderbird and WebApp how it looks. The word should correctly be shown as “Geschäftsbericht”.

Gerald

Am I really the only one with the problem?

I still have the problem of the mangled subject lines, even after updating to the very latest core-8.6.80.713_0+76-Ubuntu_16.04 (downloaded right now, but the files are from 2018-04-27).

Am I doing something wrong? I redid the config files of course, but I don’t see anything obvious. In spooler.cfg “charset_upgrade = windows-1252” was set in older versions of kopano (now it’s commented), but uncommenting this line did not resolve the problem…

Weird!

fkroeger

Hi,
I’m having the same problem.
Installed version -8.6.80.713_0+76- on ubuntu 16.04 this weekend. I changed the language to de_DE.UTF-8 in every config file I could find.

In Webapp the folder names are scrambled: e.g. Entw?rfe instead of Entwürfe
I have to run a script to rename the folders for every newly created user.

Furthermore the subject line is not quoted and therefore gets scrambled. It doesn’t matter what client I use.
I tried Kopano webapp and Outlook 2016 with active sync.
The To and From Headers are correctly quoted (?utf8?). The subject line is not.
These are the headers on the receiving end:

Subject: Test  hgf
From: =?utf-8?Q?Frank_Kr=C3=B6ger?= <xx@xxxxx.de>
To: =?utf-8?Q?xxx?= <xxx@xxxxx.de>

Subject should be: Test öäüß hgf

On the receiving end the Subject is: Test ??? hgf

I have no idea if there is some configuration missing or it is simply a bug.
Even more annoying is that we have a ciphermail gateway sitting between kopano and the internet. The scrambled subject gets even more scrambled if it is routed through ciphermail. Then there are no ? but a couple of weird symbols.

I hope this gets fixed soon or someone can give a hint what is missing.

Tia

Frank

fkroeger

I just checked a mail with Thunderbird.
The header looks like this:

Subject: Test Ã¶Ã¼Ã¤ÃŸ hgf
From: =?utf-8?Q?Frank_Kr=C3=B6ger?= <xxx@xxx.de>
To: =?utf-8?Q?info?= <xxx@xxx.de>

Thunderbird shows the subject in the list view as: Test ??? hgf
If I open the mail to read it shows Subject: Test öäüß hgf

The umlaute are encoded in the subject but not quoted.
Perhaps this could give a hint where the problem might be.

fbartels

Hi @gerald ,

I was able to reproduce this and created https://jira.kopano.io/browse/KC-1139 to follow up on this.

@fkroeger said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

In Webapp the folder names are scrambled: e.g. Entw?rfe instead of Entwürfe

This must be something else. On my testsystem I can create german folder names just fine, e.g. with kopano-admin --create-store user2 --lang de_DE.UTF-8

Gerald

Thank you for opening the bug about this fbartels!

I accidentally noticed the WebApp folder-name-problem today as well. It seems to only affect the newly created user (which logged in using Outlook 2016 via latest Z-Push). Other, older Accounts do not show those damaged foldernames for “Entwürfe” and “Gelöschte”…

So it probably is related to initial account/folder creation? And not about displaying existing folders.

fbartels

Most probably related to https://forum.kopano.io/topic/1308/using-french-locale-accentuation-not-working-in-mailbox-content/. I would propose to discuss there further.

fkroeger

Hi @fbartels,

thank you for opening the ticket.

kopano-admin --create-store user2 --lang de_DE.UTF-8

always worked for me.
The problem is following:

I create a user with:

kopano-admin -c test -p xxxx -e "test@xxx.de" -f "test"

in /etc/kopano/userscripts/createuser.d/00createstore I have the following:

kopano-admin -v --utf8 --create-store "${KOPANO_USER}" --lang "${KOPANO_LANG}"

even if I change this line to:

kopano-admin -v --utf8 --create-store "${KOPANO-USER}" --lang "de_DE.UTF-8"

the store is not created with de_DE.UTF-8 encoding.
I have to detach the store, use the command from above to create a correct store and remove the orphaned store.

@fbartels said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

Hi @gerald ,

I was able to reproduce this and created https://jira.kopano.io/browse/KC-1139 to follow up on this.

@fkroeger said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

In Webapp the folder names are scrambled: e.g. Entw?rfe instead of Entwürfe

This must be something else. On my testsystem I can create german folder names just fine, e.g. with kopano-admin --create-store user2 --lang de_DE.UTF-8

fbartels

Hi @fkroeger ,

I’ve managed to reproduce the issue and create a ticket for followup https://jira.kopano.io/browse/KC-1140

umgfoin

Hello folks,
The subject/Umlaut issue seems to be fixed for me with @bosim 's recent commit bfef0f0e9a0
.
++umgfoin

bosim

@umgfoin said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

Hello folks,
The subject/Umlaut issue seems to be fixed for me with @bosim 's recent commit bfef0f0e9a0
.
++umgfoin

Thanks for verifying! :)

Bo

Gerald

I just installed version 8.6.80.752-0+85.1 and can confirm that the Subject/Umlaut issue is fixed for me as well!

robgnu

Hello,
today I’ve upgraded to the current nightly (06.05.2018 / core-8.6.80.752_0+85-Debian_9.0-amd64) and I can’t confirm the fix. When I create a new user, I still get the ? instead of the “Umlauts” in the folder names.

I created a user with this command:

kopano-cli --create --user user1 --password "user1" --email user1@example.com --fullname "User One" --admin-level 1

robgnu

Gerald

@robgnu
My confirmation that this has been fixed was only meant to be about the corrupt subject line, not about the folder name issue.

Update: Tried the folder issue myself now because I needed to create a user anyway. Still present. User created with commandline and logged in via WebApp. Folder names corrupt. Fixed them with kopano-localize-folders -u user@domain.tld --lang de_DE.UTF-8

fbartels

@robgnu said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

I can’t confirm the fix.

We are talking about two different issues here. Kc-1139 was solved, kc-1140 is still open.

robgnu

Oh ok - sorry.

noise

hello
i have a similar problem with run

kopano-localize-folders -u user@domain.tld --lang de_DE.UTF-8

the output is:

user@domain.tld
Traceback (most recent call last):
  File "/usr/sbin/kopano-localize-folders", line 114, in <module>
    main()
  File "/usr/sbin/kopano-localize-folders", line 88, in main
    print('%s: Changing localized folder names to \"%s\"' % (user.name.decode('utf-8'), options.lang))
AttributeError: 'str' object has no attribute 'decode'

anyone have a idea of this ? running ubuntu 18.04 core-8.7.80

noise

i’m no able to solve my above problem but i’m able to have all umlaut in german language…

first detach store from user:

kopano-storeadm -Dn <user1>

add new store to user

kopano-storeadm -Cn <user1> -l de_DE.UTF-8

after this i got german names in folders like inbox > posteingang
but the umlauts are not displaying correct.
My charset of nginx is set to UTF-8

so i found in the spooler.cfg

#charset_upgrade = windows-1252

i change it to:

charset_upgrade = UTF-8

now it looks to work properly, all umlaut are displaying

thctlo

a very good point here.

dagent.cfg:#default_charset = us-ascii
ical.cfg:#default_charset = utf-8
ldap.cfg:#ldap_server_charset = utf-8
spooler.cfg:#charset_upgrade = windows-1252
unix.cfg:fullname_charset = iso-8859-15

uhhhh, why o why ?? please fix this all to UTF8

noise

ok i look at other files too…

edit: dagent.cfg charset is uncommented i leave it at this time but you have right why not all config files are to UTF-8 ?