core-8.6.80.621: German Umlauts broken in the subject line of sent mails

fkroeger

I just checked a mail with Thunderbird.
The header looks like this:

Subject: Test Ã¶Ã¼Ã¤ÃŸ hgf
From: =?utf-8?Q?Frank_Kr=C3=B6ger?= <xxx@xxx.de>
To: =?utf-8?Q?info?= <xxx@xxx.de>

Thunderbird shows the subject in the list view as: Test ??? hgf
If I open the mail to read it shows Subject: Test öäüß hgf

The umlaute are encoded in the subject but not quoted.
Perhaps this could give a hint where the problem might be.

fbartels

Hi @gerald ,

I was able to reproduce this and created https://jira.kopano.io/browse/KC-1139 to follow up on this.

@fkroeger said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

In Webapp the folder names are scrambled: e.g. Entw?rfe instead of Entwürfe

This must be something else. On my testsystem I can create german folder names just fine, e.g. with kopano-admin --create-store user2 --lang de_DE.UTF-8

Gerald

Thank you for opening the bug about this fbartels!

I accidentally noticed the WebApp folder-name-problem today as well. It seems to only affect the newly created user (which logged in using Outlook 2016 via latest Z-Push). Other, older Accounts do not show those damaged foldernames for “Entwürfe” and “Gelöschte”…

So it probably is related to initial account/folder creation? And not about displaying existing folders.

fbartels

Most probably related to https://forum.kopano.io/topic/1308/using-french-locale-accentuation-not-working-in-mailbox-content/. I would propose to discuss there further.

fkroeger

Hi @fbartels,

thank you for opening the ticket.

kopano-admin --create-store user2 --lang de_DE.UTF-8

always worked for me.
The problem is following:

I create a user with:

kopano-admin -c test -p xxxx -e "test@xxx.de" -f "test"

in /etc/kopano/userscripts/createuser.d/00createstore I have the following:

kopano-admin -v --utf8 --create-store "${KOPANO_USER}" --lang "${KOPANO_LANG}"

even if I change this line to:

kopano-admin -v --utf8 --create-store "${KOPANO-USER}" --lang "de_DE.UTF-8"

the store is not created with de_DE.UTF-8 encoding.
I have to detach the store, use the command from above to create a correct store and remove the orphaned store.

@fbartels said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

Hi @gerald ,

I was able to reproduce this and created https://jira.kopano.io/browse/KC-1139 to follow up on this.

@fkroeger said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

In Webapp the folder names are scrambled: e.g. Entw?rfe instead of Entwürfe

This must be something else. On my testsystem I can create german folder names just fine, e.g. with kopano-admin --create-store user2 --lang de_DE.UTF-8

fbartels

Hi @fkroeger ,

I’ve managed to reproduce the issue and create a ticket for followup https://jira.kopano.io/browse/KC-1140

umgfoin

Hello folks,
The subject/Umlaut issue seems to be fixed for me with @bosim 's recent commit bfef0f0e9a0
.
++umgfoin

bosim

@umgfoin said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

Hello folks,
The subject/Umlaut issue seems to be fixed for me with @bosim 's recent commit bfef0f0e9a0
.
++umgfoin

Thanks for verifying! :)

Bo

Gerald

I just installed version 8.6.80.752-0+85.1 and can confirm that the Subject/Umlaut issue is fixed for me as well!

robgnu

Hello,
today I’ve upgraded to the current nightly (06.05.2018 / core-8.6.80.752_0+85-Debian_9.0-amd64) and I can’t confirm the fix. When I create a new user, I still get the ? instead of the “Umlauts” in the folder names.

I created a user with this command:

kopano-cli --create --user user1 --password "user1" --email user1@example.com --fullname "User One" --admin-level 1

robgnu

Gerald

@robgnu
My confirmation that this has been fixed was only meant to be about the corrupt subject line, not about the folder name issue.

Update: Tried the folder issue myself now because I needed to create a user anyway. Still present. User created with commandline and logged in via WebApp. Folder names corrupt. Fixed them with kopano-localize-folders -u user@domain.tld --lang de_DE.UTF-8

fbartels

@robgnu said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

I can’t confirm the fix.

We are talking about two different issues here. Kc-1139 was solved, kc-1140 is still open.

robgnu

Oh ok - sorry.

noise

hello
i have a similar problem with run

kopano-localize-folders -u user@domain.tld --lang de_DE.UTF-8

the output is:

user@domain.tld
Traceback (most recent call last):
  File "/usr/sbin/kopano-localize-folders", line 114, in <module>
    main()
  File "/usr/sbin/kopano-localize-folders", line 88, in main
    print('%s: Changing localized folder names to \"%s\"' % (user.name.decode('utf-8'), options.lang))
AttributeError: 'str' object has no attribute 'decode'

anyone have a idea of this ? running ubuntu 18.04 core-8.7.80

noise

i’m no able to solve my above problem but i’m able to have all umlaut in german language…

first detach store from user:

kopano-storeadm -Dn <user1>

add new store to user

kopano-storeadm -Cn <user1> -l de_DE.UTF-8

after this i got german names in folders like inbox > posteingang
but the umlauts are not displaying correct.
My charset of nginx is set to UTF-8

so i found in the spooler.cfg

#charset_upgrade = windows-1252

i change it to:

charset_upgrade = UTF-8

now it looks to work properly, all umlaut are displaying

thctlo

a very good point here.

dagent.cfg:#default_charset = us-ascii
ical.cfg:#default_charset = utf-8
ldap.cfg:#ldap_server_charset = utf-8
spooler.cfg:#charset_upgrade = windows-1252
unix.cfg:fullname_charset = iso-8859-15

uhhhh, why o why ?? please fix this all to UTF8

noise

ok i look at other files too…

edit: dagent.cfg charset is uncommented i leave it at this time but you have right why not all config files are to UTF-8 ?

thctlo

no, i say this is one that kopano should fix, its crazy to have so much differnent types.

thctlo

@noise

edit: dagent.cfg charset is uncommented i leave it at this time but you have right why not all config files are to UTF-8 ?

i dont know, wait for a kopano dev to reply this.

fbartels

@thctlo said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

this is one that kopano should fix

It’s all open source, you know. If you feel you need to change something: https://stash.kopano.io/projects/KC/repos/kopanocore/browse/CONTRIBUTING.md

I doubt that it is really necessary, since there are plenty of systems out there that work fine with umlauts in subjects without the need to fiddle with charset settings in the spooler.

@noise said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

Traceback (most recent call last):

I did not yet manage to look into this, but I would expect that this is a python2 vs python3 issue. in master we recently switched everything to python3 and some scripts may still require love.