server.log: [warning] SQL [00011047] info: Try to reconnect - german umlauts

Fux

server.log:

Thu Mar 15 08:42:39 2018: [error  ] KDatabase::I_Update() query failed: REPLACE INTO properties (hierarchyid,tag,type,properties.val_ulong,properties.val_string,properties.val_bXXXry,properties.val_double,properties.val_longint,properties.val_hi,properties.val_lo) VALUES(57877,34419,30,null,'\"Wed, 14 Mar 2018 00:58:31 +0000\" \"=?UTF-8?Q?Die_AGB_von_PayPal_=C3=A4ndern_sich?=\" ((\"PayPal\" NIL \"paypal\" \"mail.paypal.de\")) ((\"PayPal\" NIL \"paypal\" \"mail.paypal.de\")) ((\"noreply@mail.paypal.de\" NIL \"noreply-HP2v4000001622204003e9cc6086e966f45e8070\" \"mail.paypal.de\")) ((NIL NIL \"XXX.XXXXXXXX\" \"arcor.de\")) NIL NIL NIL \"<HP2v4000001622204003e9cc6086e966f45e8070@mail.paypal.de>\"',null,null,null,null,null),(57877,34422,30,null,'11,17,gggruggvucftvghtrhhoucdtuddrgedtgedrtddugddvlecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqigfgfgenuceurghilhhouhhtmecufedtudenucdnofetkffnkffpifculddujedmnecujfgurhepjffkgghruffhvffftgesrgdtreertddtjeenucfhrhhomheprfgrhifrrghluceophgrhihprghlsehmrghilhdrphgrhihprghlrdguvgeqnecuffhomhgrihhnpehprgihphgrlhdruggvpdhprgihphgrlhdqtghomhhmuhhnihgtrghtihhonhdrtghomhenucfkphepudegvddrheegrddvgeegrddutdeknecurfgrrhgrmhepmhgrgihmshhgshhiiigvpedutdegkeehjeeipdhhvghlohepmhhtrgdutdeksgdrphhmgidurdgvphhslhdurdgtohhmpdhinhgvthepudegvddrheegrddvgeegrddutdekpdhmrghilhhfrhhomhepsghouhhntggvqdfjrfdvvhegtddttddttdduiedvvddvtdegtddtfegvlegttgeitdekiegvleeiiehfgeehvgektdejtdesmhgrihhlrdhprgihphgrlhdruggvpdhrtghpthhtohepihhnrgdrphhrihgvshhinhhgsegrrhgtohhrrdguvgenucevlhhushhtvghrufhiiigvpeekge',null,null,null,null,null),(57877,34423,30,null,'newsletters',null,null,null,null,null),(57877,34424,30,null,'11',null,null,null,null,null),(57877,34425,30,null,'17',null,null,null,null,null),(57877,34426,30,null,'clean',null,null,null,null,null),(57877,34427,30,null,'clean',null,null,null,null,null),(57877,34428,30,null,'t=5,a=1,l=0',null,null,null,null,null),(57877,34437,30,null,'nil',null,null,null,null,null),(57877,34448,30,null,'Low',null,null,null,null,null),(57877,26,30,null,'IPM.Note',null,null,null,null,null),(57877,55,30,null,'Die AGB von PayPal ändern sich',null,null,null,null,null),(57877,57,64,null,null,null,null,null,30653231,2448266624),(57877,59,258,null,null,'SMTP:PAYPAL@MAIL.PAYPAL.DE\0',null,null,null,null),(57877,61,30,null,'',null,null,null,null,null),(57877,63,258,null,null,'\0\0\0\0<AC>!<A9>P@<D3><EE>H<B3>^Y<FB><A7>S0D%^A\0\0\0^F\0\0\0^E\0\0\0T6MoAA6dRO+jBHPZmKrHNA==\0\0\0\0',null,null,null,null),(57877,64,30,null,'XXX XXXXXXXX',null,null,null,null,null),(57877,65,258,null,null,'\0\0\0\0<81>+^_<A4><BE><A3>^P^Y<9D>n\0<DD>^A^OT^B\0\0^A<80>P\0a\0y\0P\0a\0l\0\0\0S\0M\0T\0P\0\0\0p\0a\0y\0p\0a\0l\0@\0m\0a\0i\0l\0.\0p\0a\0y\0p\0a\0l\0.\0d\0e\0\0\0',null,null,null,null),(57877,66,30,null,'PayPal',null,null,null,null,null),(57877,79,258,null,null,'^A\0\0\0<D4>\0\0\0<D0>\0\0\0\0\0\0\0<81>+^_<A4><BE><A3>^P^Y<9D>n\0<DD>^A^OT^B\0\0^A<80>n\0o\0r\0e\0p\0l\0y\0@\0m\0a\0i\0l\0.\0p\0a\0y\0p\0a\0l\0.\0d\0e\0\0\0S\0M\0T\0P\0\0\0n\0o\0r\0e\0p\0l\0y\0-\0H\0P\02\0v\04\00\00\00\00\00\01\06\02\02\02\00\04\00\00\03\0e\09\0c\0c\06\00\08\06\0e\09\06\06\0f\04\05\0e\08\00\07\00\0@\0m\0a\0i\0l\0.\0p\0a\0y\0p\0a\0l\0.\0d\0e\0\0\0',null,null,null,null),(57877,80,30,null,'noreply@mail.paypal.de',null,null,null,null,null),(57877,81,258,null,null,'ZARAFA:XXX.XXXXXXXX@ARCOR.DE\0',null,null,null,null),(57877,87,11,1,null,null,null,null,null,null),(57877,88,11,0,null,null,null,null,null,null),(57877,89,11,1,null,null,null,null,null,null),(57877,100,30,null,'SMTP',null,null,null,null,null),(57877,101,30,null,'paypal@mail.paypal.de',null,null,null,null,null),(57877,112,30,null,'Die AGB von PayPal ändern sich',null,null,null,null,null),(57877,117,30,null,'ZARAFA',null,null,null,null,null),(57877,118,30,null,'XXX.XXXXXXXX@arcor.de',null,null,null,null,null),(57877,125,30,null,'Return-Path: <bounce-HP2v4000001622204003e9cc6086e966f45e8070@mail.paypal.de>\r\nReceived: from mail.XXXXX.dyndns.org ([::ffff:127.0.0.1]:35922)\r\n    by mail.XXXXX.dyndns.org (kopano-dagent) with LMTP;\r\n       Thu, 15 Mar 2018 08:42:39 +0100 (CET)\r\nReceived: from pop.vodafonemail.xion.oxcs.net (localhost [127.0.0.1])\r\n      by mail.XXXXX.dyndns.org (Postfix) with ESMTP id DD6EE2C0084\r\n      for <XXX.XXXXXXXX@arcor.de>; Wed, 14 Mar 2018 01:58:55 +0100 (CET)\r\nDelivered-To: 3@2616078\r\nReceived: from imap-director-4.novalocal ([10.10.2.8])\r\n     by imap-backend-25.dovecot.xion.oxcs.net with LMTP id WPt5MrlzqFqqcgAAmQmeeg\r\n        for <3@2616078>; Wed, 14 Mar 2018 00:58:33 +0000\r\nReceived: from vsmx004l.vodafonemail.xion.oxcs.net ([10.10.2.8])\r\n        by imap-director-4.novalocal with LMTP id mPlAMrlzqFqKeAAAIFweDA\r\n    ; Wed, 14 Mar 2018 00:58:33 +0000\r\nReceived: from vsmx004.vodafonemail.xion.oxcs.net (unknown [192.168.75.198])\r\n   (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\r\n (No client certificate requested)\r\n   by vsmx004l.vodafonemail.xion.oxcs.net (Postfix) with ESMTPS id 401Cyd5kjtz2Cs2w\r\n    for <XXX.XXXXXXXX@arcor.de>; Wed, 14 Mar 2018 00:58:33 +0000 (UTC)\r\nReceived: from mta-p1.oxcloud-vadesecure.net (mta-p1.oxcloud-vadesecure.net [157.97.76.182])\r\n  (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\r\n (No client certificate requested)\r\n   by vsmx001.vodafonemail.xion.oxcs.net (Postfix) with ESMTPS id 401Cyd59lNz2Cs2g\r\n     for <XXX.XXXXXXXX@arcor.de>; Wed, 14 Mar 2018 00:58:33 +0000 (UTC)\r\nReceived: from mta108b.pmx1.epsl1.com (mta108b.pmx1.epsl1.com [142.54.244.108])\r\n       (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\r\n      (No client certificate requested)\r\n   by mta-p1.oxcloud-vadesecure.net (ox1mtai64p) with ESMTPS id 1E8042334D8\r\n    for <XXX.XXXXXXXX@arcor.de>; Wed, 14 Mar 2018 00:58:30 +0000 (UTC)\r\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mail.paypal.de;\r\n       s=pp-epsilon1; t=1520989111;\r\n        bh=w9djO1VJkCOsQs7xvL80yFk5B8zvkru2slxuf5Op8iY=;\r\n    h=MIME-Version:Subject:From:To:Date:Content-Type;\r\n   b=y0/5uQGov9RI+zLq9k9W1LYJzB+RCJsC4+HnzI+ZA5QPm3KU+Siy06S9ZEEg3to88\r\n  SnwZbnqVgwaCcQ/2vEe6RwsrFLKDjZYkzzleaebs6jPJ7gvvefoStD4MyJB3mDydDJ\r\n  kRTp0zGtRAeKiuKpI7nK5fNN4e26N3ewJAqMNn6nIMTmGTpIQoF33eVKlickVx3UYc\r\n  f26KtesoVcmeHQSDLeJoCGjCLMEomNqZG1a6jGni98dVCSEPao8gu6h24mvljkkhbo\r\n  oiATTvZXEG/cCPDIDgnDmhgud7qDzeiQxJfl1xA5yplPeqG2u1sGiAqB58Y/s6IzU2\r\n  S0od/HK70nwqQ==\r\nReceived: from [10.233.18.118] ([10.233.18.118:34334])\r\n  by pc1udsmtn2n14 (envelope-from <bounce-HP2v4000001622204003e9cc6086e966f45e8070@mail.paypal.de>)\r\n   (ecelerity 3.6.9.48312 r(Core:3.6.9.0)) with ECSTREAM\r\n       id 79/95-25034-7B378AA5; Wed, 14 Mar 2018 00:58:31 +0000\r\nList-Unsubscribe: <mailto:bounce-HP2v4000001622204003e9cc6086e966f45e8070@mail.paypal.de?subject=list-unsubscribe>\r\nMessage-ID: <HP2v4000001622204003e9cc6086e966f45e8070@mail.paypal.de>\r\nMIME-Version: 1.0\r\nReply-To: \"noreply@mail.paypal.de\"\r\n <noreply-HP2v4000001622204003e9cc6086e966f45e8070@mail.paypal.de>\r\nSubject: =?UTF-8?Q?Die_AGB_von_PayPal_=C3=A4ndern_sich?=\r\nFrom: PayPal <paypal@mail.paypal.de>\r\nTo: XXX.XXXXXXXX@arcor.de\r\nDate: Wed, 14 Mar 2018 00:58:31 +0000\r\nContent-Type: multipart/alternative;\r\n boundary=\"-=Part.61cfa91.96ee24effe84afb8.1622204067c.2e24f437a38b4d2d=-\"\r\nX-VRC-SPAM-STATUS: 11,17,gggruggvucftvghtrhhoucdtuddrgedtgedrtddugddvlecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqigfgfgenuceurghilhhouhhtmecufedtudenucdnofetkffnkffpifculddujedmnecujfgurhepjffkgghruffhvffftgesrgdtreertddtjeenucfhrhhomheprfgrhifrrghluceophgrhihprghlsehmrghilhdrphgrhihprghlrdguvgeqnecuffhomhgrihhnpehprgihphgrlhdruggvpdhprgihphgrlhdqtghomhhmuhhnihgtrghtihhonhdrtghomhenucfkphepudegvddrheegrddvgeegrddutdeknecurfgrrhgrmhepmhgrgihmshhgshhiiigvpedutdegkeehjeeipdhhvghlohepmhhtrgdutdeksgdrphhmgidurdgvphhslhdurdgtohhmpdhinhgvthepudegvddrheegrddvgeegrddutdekpdhmrghilhhfrhhomhepsghouhhntggvqdfjrfdvvhegtddttddttdduiedvvddvtdegtddtfegvlegttgeitdekiegvleeiiehfgeehvgektdejtdesmhgrihhlrdhprgihphgrlhdruggvpdhrtghpthhtohepihhnrgdrphhrihgvshhinhhgsegrrhgtohhrrdguvgenucevlhhushhtvghrufhiiigvpeekge\r\nX-VRSPAM-STATE: newsletters\r\nX-VR-SPAM-STATE: 11\r\nX-VR-SPAM-SCORE: 17\r\nX-VR-SPF-RESULT: nil\r\nX-Advert: Low\r\nX-Virus-Status-VR: clean\r\nX-Virus-Status-CA: clean\r\nX-VRC-SPAM-STATUS: 11,17,gggruggvucftvghtrhhoucdtuddrgedtgedrtddugddvlecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqigfgfgenuceurghilhhouhhtmecufedtudenucdnofetkffnkffpifculddujedmnecujfgurhepjffkgghruffhvffftgesrgdtreertddtjeenucfhrhhomheprfgrhifrrghluceophgrhihprghlsehmrghilhdrphgrhihprghlrdguvgeqnecuffhomhgrihhnpehprgihphgrlhdruggvpdhprgihphgrlhdqtghomhhmuhhnihgtrghtihhonhdrtghomhenucfkphepudegvddrheegrddvgeegrddutdeknecurfgrrhgrmhepmhgrgihmshhgshhiiigvpedutdegkeehjeeipdhhvghlohepmhhtrgdutdeksgdrphhmgidurdgvphhslhdurdgtohhmpdhinhgvthepudegvddrheegrddvgeegrddutdekpdhmrghilhhfrhhomhepsghouhhntggvqdfjrfdvvhegtddttddttdduiedvvddvtdegtddtfegvlegttgeitdekiegvleeiiehfgeehvgektdejtdesmhgrihhlrdhprgihphgrlhdruggvpdhrtghpthhtohepihhnrgdrphhrihgvshhinhhgsegrrhgtohhrrdguvgenucevlhhushhtvghrufhiiigvpeekge\r\nX-VRSPAM-STATE: newsletters\r\nX-VRC-POLICY-STATUS: t=5,a=1,l=0',null,null,null,null,null),(57877,3097,258,null,null,'\0\0\0\0<81>+^_<A4><BE><A3>^P^Y<9D>n\0<DD>^A^OT^B\0\0^A
<80>P\0a\0y\0P\0a\0l\0\0\0S\0M\0T\0P\0\0\0p\0a\0y\0p\0a\0l\0@\0m\0a\0i\0l\0.\0p\0a\0y\0p\0a\0l\0.\0d\0e\0\0\0',null,null,null,null),(57877,3098,30,null,'PayPal',null,null,null,null,null),(57877,3101,258,null,null,'SMTP:PAYPAL@MAIL.PAYPAL.DE\0',null,null,null,null),(57877,3102,30,null,'SMTP',null,null,null,null,null),(57877,3103,30,null,'paypal@mail.paypal.de',null,null,null,null,null),(57877,3586,30,null,'',null,null,null,null,null),(57877,3587,30,null,'',null,null,null,null,null),(57877,3588,30,null,'XXX XXXXXXXX',null,null,null,null,null),(57877,3590,64,null,null,null,null,null,30653489,826704256),(57877,3591,3,2,null,null,null,null,null,null),(57877,3592,3,956419,null,null,null,null,null,null),(57877,4096,30,null,'XXX XXXXXXXX – wir aktualisieren die PayPal-Richtlinien.                Online ansehen \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n      \r\n \r\n \r\n \r\n \r\n PayPal informiert: \r\n \r\n Neue Nutzungsbedingungen und Datenschutzgrundsätze ab 25. Mai 2018 \r\n \r\n Guten Tag XXX XXXXXXXX, \r\n \r\n wir arbeiten für Sie fortlaufend daran, PayPal sicherer zu machen. Das bringt auch von Zeit zu Zeit Änderungen an den AGB mit sich: Zum 25. Mai 2018 werden wir unsere Nutzungsbedingungen
Thu Mar 15 08:42:39 2018: [warning] SQL [00015791] info: Try to reconnect

dagent.log:

Thu Mar 15 08:42:39 2018: [error  ] [11036] Unable to commit message: disk error (80040116)
Thu Mar 15 08:42:39 2018: [error  ] [11036] Unable to deliver message to "XXX.XXXXXXX@arcor.de": disk error (80040116)
Thu Mar 15 08:42:39 2018: [error  ] [11036] Unable to deliver all messages for server "default:"

Most messages are delivered correctly. At this moment postqueue -p shows two messages waiting.

I thought it where a problem with the umlauts because the last line in server.log contains some umlauts.

fbartels

Can you also have a look into the logging of MariaDB to see why they are closing the connection?

We investigated a similar case recently and for most systems this was resolved by updating MariaDB to have a higher innodb_log_file_size (by default this is only 5MB if I recall correctly).

Fux

I’ve set: innodb_log_file_size = 400M (before the logfiles had a size of 48 MiB).

The error is still the same.

fbartels

Ah. I think I mixed this up. Can you check what you have set as max_allowed_packet?

also please be aware that Debian 9 has a quite overcomplicated way to source config files for MariaDB, so make sure that its not overridden afterwards:

# Import all .cnf files from configuration directory
!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mariadb.conf.d/

klausf

@fbartels Increasing MariaDB “max_allowed_packet” does not change anything. I can repoduce the problem with the same paypal e-mail.
I already described the problem in servicedesk ticket KS-40262 and also uploaded the email which causes the bug.

Fux

I increased max_allowed_packet from 16M to 100M - but had no success.

There are still some mails that cannot be delivered. The server.log says “Try to reconnect”. Shortly before this I see a fragment of the mail with Umlauts in it.

fbartels

@klausf I’ve asked our support to create a separate Jira ticket for you issue. I gave your mail a try and was able to successfully deliver it to a Debian 9 machine in our test environment. Therefore my only conclusion for now is that this has to be with tuning parameters of the database.

See https://kb.kopano.io/display/WIKI/Basic+MySQL+tuning for some general tuning parameters.

klausf

@fbartels I created the issue in my partner portal account and added private information in server logs and email attachment. Please do not make such information public available.
Your collegue Mr. Stradinger was able to reproduce the issue in his test environment. Maybe you can find out the differences between your machine configurations.

Regards Klaus

fbartels

@klausf said in server.log: [warning] SQL [00011047] info: Try to reconnect - german umlauts:

Please do not make such information public available.

what kind of private information do you see in the linked ticket?

edit: on second look there were some mail domains. i just removed them from the description.

klausf

@fbartels I have been assured by your sales department that information I enter via the partner portal will never be made public. Which information in this case is private for me is not relevant. So again my request not to make the ticket public. There is already a support case for the topic and the ticket makes no sense at all.

fbartels

By default all jira tickets should be public so others can benefit from them as well. WIll talk to our sales guys to clear up the confusion.

genesis74

During setup of Kopano in a Univention environment I ran into exactly the same problem when migrating the email data. Infact, it is several (but not all) paypal emails regarding changes in their general terms. These mails are very similar, but some break the database connection, others don’t. It does not matter if I try to import via IMAP (kopano-gateway) or Outlook MAPI (kopano-server) oder fetchmail (kopano-dagent), the problematic emails always break the database connection with server.log entry as described by Fux.

Kopano 8.4.5.0-0+33.2 (current under UCS)
MariaDB 10.1.26-0+deb9u1
Edit:
UCS 4.3-0 errata9
Optimizing the database makes no difference. Changing character set of database from latin1 to utf8 does not help as well. Issue is affecting about 1% of my emails. All affected mails are quite large and have extensive html code in their body (i.e. Paypal emails, but after more investigation also other mails are affected).

mariadb has two libs on offer (libmariadb-2.3.2, libmariadbclient-10.1) of which the former is the broken one (fixed in -3.0, but UCS does not ship that). We switched it out now.