core-8.6.80.621: German Umlauts broken in the subject line of sent mails


  • Kopano

    yeah. no, this is 100% not what i meant. my GUESS is that the code in the script you are trying to run is not compatible with python3 (which is the interpreter he tries to execute it with because of the shebang). You will still need python2 on your system (and python2-kopano or however the file is called in your distribution). simply run the script with python2 instead of python3. if you don’t want to fiddle with the script, then a python2 /usr/sbin/kopano-localize-folders .. probably does the trick as well.



  • ah ok my problem is python 3 not 2 ok i will try… thanks

    Edit:

    ok i solved with install

    apt install python-kopano
    

    with the command for all users:

    python2 /usr/sbin/kopano-localize-folders --lang de_DE.UTF-8
    

    the user are changed:

    <user1>: Changing localized folder names to "de_DE.UTF-8"
    

    thanks for the hint.



  • Hmmm but strange now i lost the umlauts again with the above…

    Edit:

    Now i try a few things also changing from
    DE.UTF-8 to de.utf8

    my settings are:

    $ locale
    LANG=de_DE.UTF-8
    LANGUAGE=
    LC_CTYPE="de_DE.UTF-8"
    LC_NUMERIC="de_DE.UTF-8"
    LC_TIME="de_DE.UTF-8"
    LC_COLLATE="de_DE.UTF-8"
    LC_MONETARY="de_DE.UTF-8"
    LC_MESSAGES="de_DE.UTF-8"
    LC_PAPER="de_DE.UTF-8"
    LC_NAME="de_DE.UTF-8"
    LC_ADDRESS="de_DE.UTF-8"
    LC_TELEPHONE="de_DE.UTF-8"
    LC_MEASUREMENT="de_DE.UTF-8"
    LC_IDENTIFICATION="de_DE.UTF-8"
    LC_ALL=
    
    $ locale -a
    C
    C.UTF-8
    de_AT.utf8
    de_BE.utf8
    de_CH.utf8
    de_DE.utf8
    de_IT.utf8
    de_LI.utf8
    de_LU.utf8
    POSIX
    

    i try to use booth config (old and new) alone and booth togheter now are booth active:

    $ cat /etc/default/kopano
    KOPANO_LOCALE="de_DE.UTF-8"
    KOPANO_USERSCRIPT_LOCALE="de_DE.UTF-8"
    
    $ cat /etc/kopano/admin.cfg
    # The language for folders in newly-created stores, specified as a
    # locale identifier ("en_US", "de_DE", etc.)
    default_store_locale = de_DE.UTF-8
    #server_socket = default:
    #sslkey_file = some.pem
    #sslkey_pass = magic
    

    but a email is watching like:
    Header:

    f?r die ?mis zum ?ss? > ?? ?? ?? ? ? ? ?
    User <user@gmail.com>
    Gesendet:Mittwoch 19 Dezember 2018 20:44
    An:
    User1 <user1@domain.com>
    

    Email Text:

    Test inside:
    für die ömis zum ässä > Ää Öö Üü è é ç à
    

    inside the email the umlauts are correct but not on header and folders

    Note:
    Always after a full reboot i can see that displaying is correct but later not

    30 sec to 1 min later

    Strange…
    all is up to date newest version ubuntu 18.04

    Edit 2:
    if set in
    /etc/default/locale
    to
    LANG=de_DE.UTF-8
    after reboot the displaying is correct fro 30-60 seconds…

    if set to:
    de_DE.UTF8
    after reboot the displaying in incorrect from first time reloading…
    and also with a emtpy file…


  • Kopano

    Ah, I missed your edits until now. The “issue” is not with the charsets that you set in any of the kopano configuration files, and also not with the locale you define when creating the store. The thing you need to look into is the charset that php uses.



  • ok that means i need to look in nginx config or php ?
    the header of nginx say it is utf-8

    but one thing is strange here or normal i don’t know:

    with:

    wget --server-response -O /dev/null https://www.domain.com/webapp
    

    the output is:
    Content-Type: text/html; charset=utf-8

    but with:

    curl -I https://domain.com/webapp
    

    the output is:
    Content-Type: text/html

    what is correct ? a check with a online tool means also it is utf-8

    phpini from fpm is:
    default_charset = "UTF-8"


  • Kopano

    @noise said in core-8.6.80.621: German Umlauts broken in the subject line of sent mails:

    ok that means i need to look in nginx config or php ?

    yes, php



  • hmmm strange i can’t find anything… charset is utf-8 in php

    the complete php output is too big to post here…
    so i create a pdf link:
    https://de.scribd.com/document/396095828/phpinfo



  • If it is solved in Kopano it doen’t mean it is solved at the recepient. See
    https://en.wikipedia.org/wiki/International_email#UTF-8_headers



  • its not definitely postfix because other mail servers i don’t have this.
    since it sometimes synonymous with the reload occurs, I think, it’s already php but not found yet.

    here is also my header!
    0_1545343226857_Screenshot.png



  • i look again for my problem but not really found, i check also php and remove some modules to testing but always the same
    the only one there is no charset is here:

    $ curl -I https://www.domain.example/webapp
    HTTP/1.1 301 Moved Permanently
    Server: nginx
    Date: Mon, 07 Jan 2019 10:27:11 GMT
    Content-Type: text/html
    Content-Length: 178
    Location: https://www.domain.example/webapp/
    Connection: keep-alive
    Expires: Wed, 06 Feb 2019 10:27:11 GMT
    Cache-Control: max-age=2592000
    Cache-Control: public, must-revalidate, proxy-revalidate
    Strict-Transport-Security: max-age=15768000; includeSubdomains; preload
    X-Frame-Options: SAMEORIGIN
    X-Content-Type-Options: nosniff
    X-XSS-Protection: 1; mode=block
    

    the adress: https://www.domain.example/webapp
    is the only where has no charset, but it’s a redirect to
    https://www.domain.example/webapp/
    and here we have the charset…

    $ curl -I https://www.domain.example/webapp/
    HTTP/1.1 200 OK
    Server: nginx
    Date: Mon, 07 Jan 2019 10:27:16 GMT
    Content-Type: text/html; charset=utf-8
    Connection: keep-alive
    Vary: Accept-Encoding
    Set-Cookie: __Secure-KOPANO_WEBAPP=g9cv0gsd2dh1tlc5ofo40fcpe3; path=/; secure; HttpOnly
    Expires: Wed, 06 Feb 2019 10:27:16 GMT
    Cache-Control: max-age=2592000
    Pragma: no-cache
    Set-Cookie: __Secure-encryption-store-key=51f94dc3cfbafd03948338ced240a0b9; path=/; secure; HttpOnly
    Cache-Control: public, must-revalidate, proxy-revalidate
    Strict-Transport-Security: max-age=15768000; includeSubdomains; preload
    X-Frame-Options: SAMEORIGIN
    X-Content-Type-Options: nosniff
    X-XSS-Protection: 1; mode=block
    

    i don’t know if that is my problem ?
    any hint’s are welcome



  • @ noise
    I am running under Debian 9.6 with PHP7.2 from the external “sury” Repo. I got the same problems. With a fist login in WebApp all looks fine for the german ‘umlaute’ in foldernames. German “Entwürfe” looks like “Entwürfe”. With the second login of the same user “Entwürfe” goes to “Entw?rfe”. — I get the same results as you with the different curl-Querys.
    In Mails, Mail-Headers, etc. an in all other forms of using KOPANO (imap, outlook -mapi, outlook -z-push) the foldernames and all other use of “öü” is displayed right!
    I use another Kopano-Installation with Debian 8 and php 5. Testing this install, i got the same results with your curl querys. Therefore the problem may be in an other context.
    Because of the first login, when everything looks allright i will search whats happens up to the second login.

    any hint’s are welcome



  • If folder names change over time, sounds like data is transferred into the DB in a lossy fashion (and the only reason it shows ü in the first round is because the in-memory cache keeps it in its original form… until such a time it gets evicted).



  • Hi everyone

    i solved my problem! It was made by my own. All Ü’s and Ö’s are stable back again.
    Shamefully the reason was a miss-configuration of the Apache server! No php Problem.
    The error results out of a not enabled “javascript-common.conf” and some enabled mods with conflicts.

    Härzlichen Dönk !!!



  • Hey zara-kopa,

    Could you provide details where you solved the problem in your apache config?
    I got the same error on my fresh install, and I just can’t figure out why.

    Thanks a lot and greetings from Nuremberg ;)
    Christoph.



  • @chrizzibaer

    Here are the enabled CONFS and MODS of my apache2

    1_1547981822034_Bildschirmfoto 2019-01-20 um 11.54.36.jpg 0_1547981822033_Bildschirmfoto 2019-01-20 um 11.55.00.jpg

    With these -enabled it’s worked for me. The use of php7.2-fpm is still in progress!
    nice weekend from the Ruhrpott



  • @zara-kopa i’m running at nginx and also in fpm and still have the problem…
    it is possible that comes from mysql (mariabd)
    The Kollation of kopano innodb is: utf8mb4_general_ci ?


Log in to reply