8.7.80."307-424..." missing mails in different imap-clients

irrlicht

Hi,
just installed 8.7.80.307 and there are missing mails in different imap clients, tested with eM Client, Thunderbird and K-9 Mail. All mails are in Webapp.
If i delete the konfiguration in thunderbird and make a new one, the mails are there, in eM Client no difference.
is this a known bug?

fbartels

Hi @irrlicht ,

since it’s not really related to the beta I only approved this post (new users have to be approved once to prevent spam).

No, not a known issue. More details/logging would be interesting.

irrlicht

Mails from a special Server (our trouble-ticket-system) are visible in Thunderbird Windows/Linux
Not visible in eM-Client Windows and MailDroid.
Mails from cli on this Server are visble in Thunderbird and Maildroid, not in eM-Client.

My guess: kopano-gateway or dagent, perhaps postfix 3 (old server was 2)

Could i write per mail in german with logfiles? debug-logging is activated
Don’t want to send logfiles to this forum.

irrlicht

It’s possible a problem with the content-type.

Testcommand from linux-cli arrives at eM-Client per kopano-gateway
mail -s “emShit”"$1" -a “Content-Type: text/plain; charset=UTF-8” testuser@example.com <<< testtext

without content-type not:
mail -s “emShit”"$1" testuser@example.com <<< testtext

It’s available per imap, all other imap-clients have no problems with this.
Worked before with old Zarafa

What can i do?

fbartels

@irrlicht said in 8.7.80.307 missing mails in different imap-clients:

Could i write per mail in german with logfiles? debug-logging is activated
Don’t want to send logfiles to this forum.

Private support is available for customers with a valid subscription. If required this is also possible in german.

Price information: https://kopano.com/pricing/
Tickets can be opened at: https://portal.kopano.com/sdtickets/list

Instead of logs it is probably more helpful to have such a message as a raw message. (you can add log_raw_message = yes to your dagent.cfg for this, see https://stash.kopano.io/projects/KC/repos/kopanocore/browse/installer/linux/dagent.cfg#36)

irrlicht

Thanks Felix,
i’m not a customer, we only use Kopano. :)
i’m a teacher and i also teach my students Kopano as an example for Groupware in my classes in the Servercourses with Linux in professional school. Course of Studies Fachinformatiker Systemintegration

I thougt the problem would also be interesting for developers of beta.

irrlicht

log_raw_message=on in dagent.cfg
Mail from Ticketsystem:
OK in Thunderbird, not available in eM Client und MailDroid:
Names and Domain changed

Return-Path: <schmitt@wvs-heidelberg.de>
Received: from wvs-heidelberg.de ([::ffff:127.0.0.1]:45836)
	by kopano_www_km36022-01 (kopano-dagent) with LMTP;
	Sat, 29 Dec 2018 13:38:55 +0100 (CET)
Received: from roundup.wvs-heidelberg.de (unknown [46.227.215.83])
	by wvs-heidelberg.de (Postfix) with ESMTP id C3AA53B3C70;
	Sat, 29 Dec 2018 13:38:55 +0100 (CET)
Received: from intern.wvs-heidelberg.de (localhost [127.0.0.1])
	by roundup.wvs-heidelberg.de (Postfix) with ESMTP id 8D8883C037A;
	Sat, 29 Dec 2018 13:38:55 +0100 (CET)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Reply-To: Importtesttracker <issue_tracker@roundup.wvs-heidelberg.de>
Message-Id: <1546087135.56.0.0342536634962.issue843@roundup.wvs-heidelberg.de>
X-Roundup-issue-status: chatting
X-Roundup-issue-priority: critical
In-Reply-To: <1546025630.22.0.112305386028.issue843@roundup.wvs-heidelberg.de>
Content-Transfer-Encoding: quoted-printable
Subject: [issue843] Testticket fuer das IMAP-Problem
To: schmitt.peter@wvs-heidelberg.de, maier@wvs-heidelberg.de,
 schneider@wvs-heidelberg.de
From: schmitt <issue_tracker@roundup.wvs-heidelberg.de>
Date: Sat, 29 Dec 2018 11:38:55 -0100
Precedence: bulk
X-Roundup-Name: Importtesttracker
X-Roundup-Loop: hello
X-Roundup-Version: 1.4.20
Message-Id: <1546087135.56.0.0342536634962.issue843@roundup.wvs-heidelberg.de>
Reply-To: Importtesttracker <issue_tracker@roundup.wvs-heidelberg.de>
In-Reply-To: <1546025630.22.0.112305386028.issue843@roundup.wvs-heidelberg.de>

schmitt <schmitt.peter@wvs-heidelberg.de> merkte an:
Test aus Ticketsystem
_________________________________________________________
Importtesttracker <issue_tracker@roundup.wvs-heidelberg.de>
<https://roundup.wvs-heidelberg.de/issue843>

irrlicht

Mail from t-online, same problem - not in eM-Client and not in MailDroid.
Thunderbird and Webapp perfect.

Ret```

urn-Path: <hanswurs@t-online.de>
Received: from wvs-heidelberg.de ([::ffff:127.0.0.1]:44650)
	by kopano_www_km36022-01 (kopano-dagent) with LMTP;
	Sat, 29 Dec 2018 13:28:43 +0100 (CET)
Received: from mailout11.t-online.de (mailout11.t-online.de [194.25.134.85])
	by wvs-heidelberg.de (Postfix) with ESMTPS id 82E433B3C70
	for <schneider@wvs-heidelberg.de>; Sat, 29 Dec 2018 13:28:43 +0100 (CET)
Received: from fwd30.aul.t-online.de (fwd30.aul.t-online.de [172.20.26.135])
	by mailout11.t-online.de (Postfix) with SMTP id 3C716425941E
	for <schneider@wvs-heidelberg.de>; Sat, 29 Dec 2018 13:28:43 +0100 (CET)
Received: from spica06.aul.t-online.de (TvrP-ZZrrhgSdKieGN9UDg7rDjf03f84ac8qBnz+IMcoo-KBTqjwr1ZIAHsDgyCQoZ@[172.20.102.123]) by fwd30.aul.t-online.de
	with esmtp id 1gdDjE-0fDxaq0; Sat, 29 Dec 2018 13:28:32 +0100
Received: from 46.223.239.167:62278 by cmpweb10.aul.t-online.de with HTTP/1.1 (Lisa V5-5-1-0.14719 on API V5-19-0-0)
Received: from 172.20.102.137:59863 by spica06.aul.t-online.de:8080; Sat, 29 Dec 2018 13:28:32 +0100 (CET)
Date: Sat, 29 Dec 2018 13:28:32 +0100 (CET)
From: "hanswurs@t-online.de" <hanswurs@t-online.de>
Sender: "hanswurs@t-online.de" <hanswurs@t-online.de>
Reply-To: "hanswurs@t-online.de" <hanswurs@t-online.de>
To: "schneider@wvs-heidelberg.de" <schneider@wvs-heidelberg.de>
Message-ID: <1546086512624.2010385.895fa34a42662b380d4f80a8edf1368fee1e2437@spica.telekom.de>
Subject: test1 an schneider von t-online
MIME-Version: 1.0
Content-Type: multipart/alternative; 
	boundary="----=_Part_2491006_1437171440.1546086512701"
Importance: normal
X-MSMail-Priority: normal
X-Priority: 3
X-UMS: email
X-ID: TvrP-ZZrrhgSdKieGN9UDg7rDjf03f84ac8qBnz+IMcoo-KBTqjwr1ZIAHsDgyCQoZ@t-dialin.net
X-TOI-MSGID: 4acb1054-9a73-4a8b-b00e-e0baf9265793

------=_Part_2491006_1437171440.1546086512701
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

test1
=EF=BB=BF
------=_Part_2491006_1437171440.1546086512701
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=
=3DUTF-8"/><title></title></head><body><p style=3D"margin: 0px;"><span styl=
e=3D"font-family:arial,helvetica,sans-serif;font-size:10pt;color:#000000;te=
xt-decoration:none">test1<br /></span></p>=EF=BB=BF</body></html>
------=_Part_2491006_1437171440.1546086512701--

irrlicht

MailDroid on Android
Search Subject “issue” online (Subject from missing mails from trouble ticket system) , not in local phonecache.
Logs from Kopanogateway shows the discoveries, MailDroid can’t open them. eM Client similar problem, Thunderbird no problems,
Searcherror:

Details

<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><report><board>MT6582</board><brand>Lenovo</brand><details>Create messages list screen.</details><domain>wvss-mannheim.de</domain><locale>Deutsch (Deutschland)</locale><model>Lenovo A7600-F</model><package_name>com.maildroid</package_name><package_version>4.91</package_version><protocol>imap</protocol><stack>javax.mail.MessagingException: Failed to load IMAP envelope
at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1238)
at com.sun.mail.imap.IMAPMessage.getSubject(IMAPMessage.java:345)
at com.maildroid.second.ae.a(SessionUtils.java:48)
at com.maildroid.service.o.c(SessionControllerDispatcher.java:107)
at com.maildroid.service.o.a(SessionControllerDispatcher.java:40)
at com.maildroid.service.n.a(SessionController.java:100)
at com.maildroid.service.p.b(SessionControllerWithState.java:99)
at com.maildroid.service.p.a(SessionControllerWithState.java:52)
at com.maildroid.service.m.a(RemoteMailService.java:139)
at com.maildroid.m.e.a(MailService.java:43)
at com.maildroid.bk.c.a(TasksExecutor.java:41)
at com.maildroid.bk.d$1.run(TasksExecutorThread.java:18)
at com.flipdog.commons.u.a$1.run(ThreadUtils.java:46)
at java.lang.Thread.run(Thread.java:761)
</stack><uuid>6793fec8-3f71-4e6c-9591-a8ba47374707</uuid><version_incremental>18c2428f1f</version_incremental><version_release>7.1.2</version_release><version_sdk>25</version_sdk></report>

irrlicht

I build a second kopano-server on a different machine. Mails from t-online are invisiible in different imap-clients, thunderbird is ok, eM Client not.

Could somebody test this please?

fbartels

So on the second server the behaviour is different?

Will check this out sometime in the beginning of next year.

irrlicht

Happy new year.

@fbartels said in 8.7.80.307 missing mails in different imap-clients:

So on the second server the behaviour is different?

No, same behaviour. T-online mails are hidden on MailDroid and eM-Client. Thunderbird ist ok.

irrlicht

Hi,
same problems with nightly kopano-gateway 8.7.80.402.1c7f107ba-0+44.1 on the testserver, it think it’s a problem with kopano-gateway.
Comment from another forum about the javaerror: “server seems to be broken”.

greets from germany

by the way, upgrade to 8.7.80.402 php-mapi-error:

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/mapi.so' - /usr/lib/php/20151012/mapi.so: undefined symbol: _Z25zif_mapi_freebusy_openmsgP18_zend_execute_dataP12_zval_struct in Unknown on line 0
[Wed Jan 02 19:31:21.077162 2019] [mpm_prefork:notice] [pid 12493] AH00163: Apache/2.4.25 (Debian) OpenSSL/1.0.2q configured -- resuming normal operations

irrlicht

php-mapi-error ist fixed with 8.7.80.424, t-online-mails still invisible in MailDroid and eM Client.

Does anybody have the same problems?

fbartels

@irrlicht said in 8.7.80."307-424..." missing mails in different imap-clients:

Worked before with old Zarafa

I did not see this bit before. Which Zarafa version are you speaking about?

We were able to reproduce the email not showing up in emclient with the t-online mail, but not with the mail commands you posted.

irrlicht

Hi Felix and thank you verry much for thist post,

I think it was last Zarafa-Version: 7.2.6.10 on Debian 8 and the mail commands perhaps only work from this one Server, old postfix 2. This machine runs our trouble ticket system. I will have a try from another one.

@fbartels
Edith: It’s strange, from the cli of the trouble ticket machine with postfix 2.11.3-1+deb8u2 without content-type is invisible in eM Client and in MailDroid, with content-type is ok, from other hosts all are ok, with or without.

Shellscript 1:
mail -s “emShit”"$1" user@example.com <<< testtext

Shellscript 2:
mail -s “emShit”"$1" -a “Content-Type: text/plain; charset=utf-8” user@example.com <<< testtext

irrlicht

Strange, in thunderbird are all missing mails from eM-Client visible, but a few others have no topics and no sender. Can’t reproduce them, a few of the missing parts are available a few hours later. In other thunderbird versions are only senders missing.

irrlicht

Are we the only ones with this imap problem? Problem exists with stable 8.6.9 too.

fbartels

for reference, the ticket to follow for this would be https://jira.kopano.io/browse/KC-1375. In our tests we also saw that this occurs with 8.6 as well.

I find it interesting that during the list command the message is shown, but then not picked up by the imap client later on.

Currently busy with a customer escalation, will later spent some more time on this.

irrlicht

I think the problem is fixed in 8.7.0 and in nightly 8.7.80.529

New mails from problematic servers are shown, old ones not.

kopano-dbadm kc-1375 in old installation throws errors

kopano-dbadm kc-1375
Tue Jan 29 20:18:22 2019: [notice ] kc1375: purging problematic-looking IMAP envelopes...
Tue Jan 29 20:18:25 2019: [error  ] 0x55b9bbb977d0: SQL Failed: BIGINT UNSIGNED value is out of range in '(`kopano`.`p`.`tag` - 34049)', Query: "DELETE p FROM properties AS p INNER JOIN names AS n ON p.tag-34049=n.id WHERE n.guid=0x08F1F5003F8EC746AF725E201C2349E7 AND n.nameid=1 AND p.type=30 AND p.val_string LIKE '% (((%))) %'"
Tue Jan 29 20:18:25 2019: [error  ] KDatabase::I_Update() query failed: "BIGINT UNSIGNED value is out of range in '(`kopano`.`p`.`tag` - 34049)'", query: DELETE p FROM properties AS p INNER JOIN names AS n ON p.tag-34049=n.id WHERE n.guid=0x08F1F5003F8EC746AF725E201C2349E7 AND n.nameid=1 AND p.type=30 AND p.val_string LIKE '% (((%))) %'
Tue Jan 29 20:18:25 2019: [error  ] dbadm: action failed: disk error (80000007)

on a new testserver it works

kopano-dbadm kc-1375
Tue Jan 29 20:24:38 2019: [notice ] kc1375: purging problematic-looking IMAP envelopes...
Tue Jan 29 20:24:40 2019: [notice ] kc1375: deleted 121 rows.

@fbartels : “purge” means it deletes the mails in cache or in database?