8.7.80."307-424..." missing mails in different imap-clients

irrlicht

Thanks Felix,
i’m not a customer, we only use Kopano. :)
i’m a teacher and i also teach my students Kopano as an example for Groupware in my classes in the Servercourses with Linux in professional school. Course of Studies Fachinformatiker Systemintegration

I thougt the problem would also be interesting for developers of beta.

irrlicht

log_raw_message=on in dagent.cfg
Mail from Ticketsystem:
OK in Thunderbird, not available in eM Client und MailDroid:
Names and Domain changed

MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Reply-To: Importtesttracker <issue_tracker@roundup.wvs-heidelberg.de>
Message-Id: <1546087135.56.0.0342536634962.issue843@roundup.wvs-heidelberg.de>
In-Reply-To: <1546025630.22.0.112305386028.issue843@roundup.wvs-heidelberg.de>
Content-Transfer-Encoding: quoted-printable
Subject: [issue843] Testticket fuer das IMAP-Problem
To: schmitt.peter@wvs-heidelberg.de, maier@wvs-heidelberg.de,
 schneider@wvs-heidelberg.de
From: schmitt <issue_tracker@roundup.wvs-heidelberg.de>
Date: Sat, 29 Dec 2018 11:38:55 -0100
Precedence: bulk
Message-Id: <1546087135.56.0.0342536634962.issue843@roundup.wvs-heidelberg.de>
Reply-To: Importtesttracker <issue_tracker@roundup.wvs-heidelberg.de>
In-Reply-To: <1546025630.22.0.112305386028.issue843@roundup.wvs-heidelberg.de>

schmitt <schmitt.peter@wvs-heidelberg.de> merkte an:
Test aus Ticketsystem
_________________________________________________________
Importtesttracker <issue_tracker@roundup.wvs-heidelberg.de>
<https://roundup.wvs-heidelberg.de/issue843>

irrlicht

Mail from t-online, same problem - not in eM-Client and not in MailDroid.
Thunderbird and Webapp perfect.

Ret```

urn-Path: <hanswurs@t-online.de>
Received: from wvs-heidelberg.de ([::ffff:127.0.0.1]:44650)
	by kopano_www_km36022-01 (kopano-dagent) with LMTP;
	Sat, 29 Dec 2018 13:28:43 +0100 (CET)
Received: from mailout11.t-online.de (mailout11.t-online.de [194.25.134.85])
	by wvs-heidelberg.de (Postfix) with ESMTPS id 82E433B3C70
	for <schneider@wvs-heidelberg.de>; Sat, 29 Dec 2018 13:28:43 +0100 (CET)
Received: from fwd30.aul.t-online.de (fwd30.aul.t-online.de [172.20.26.135])
	by mailout11.t-online.de (Postfix) with SMTP id 3C716425941E
	for <schneider@wvs-heidelberg.de>; Sat, 29 Dec 2018 13:28:43 +0100 (CET)
Received: from spica06.aul.t-online.de (TvrP-ZZrrhgSdKieGN9UDg7rDjf03f84ac8qBnz+IMcoo-KBTqjwr1ZIAHsDgyCQoZ@[172.20.102.123]) by fwd30.aul.t-online.de
	with esmtp id 1gdDjE-0fDxaq0; Sat, 29 Dec 2018 13:28:32 +0100
Received: from 46.223.239.167:62278 by cmpweb10.aul.t-online.de with HTTP/1.1 (Lisa V5-5-1-0.14719 on API V5-19-0-0)
Received: from 172.20.102.137:59863 by spica06.aul.t-online.de:8080; Sat, 29 Dec 2018 13:28:32 +0100 (CET)
Date: Sat, 29 Dec 2018 13:28:32 +0100 (CET)
From: "hanswurs@t-online.de" <hanswurs@t-online.de>
Sender: "hanswurs@t-online.de" <hanswurs@t-online.de>
Reply-To: "hanswurs@t-online.de" <hanswurs@t-online.de>
To: "schneider@wvs-heidelberg.de" <schneider@wvs-heidelberg.de>
Message-ID: <1546086512624.2010385.895fa34a42662b380d4f80a8edf1368fee1e2437@spica.telekom.de>
Subject: test1 an schneider von t-online
MIME-Version: 1.0
Content-Type: multipart/alternative; 
	boundary="----=_Part_2491006_1437171440.1546086512701"
Importance: normal
X-MSMail-Priority: normal
X-Priority: 3
X-UMS: email
X-ID: TvrP-ZZrrhgSdKieGN9UDg7rDjf03f84ac8qBnz+IMcoo-KBTqjwr1ZIAHsDgyCQoZ@t-dialin.net
X-TOI-MSGID: 4acb1054-9a73-4a8b-b00e-e0baf9265793

------=_Part_2491006_1437171440.1546086512701
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

test1
=EF=BB=BF
------=_Part_2491006_1437171440.1546086512701
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=
=3DUTF-8"/><title></title></head><body><p style=3D"margin: 0px;"><span styl=
e=3D"font-family:arial,helvetica,sans-serif;font-size:10pt;color:#000000;te=
xt-decoration:none">test1<br /></span></p>=EF=BB=BF</body></html>
------=_Part_2491006_1437171440.1546086512701--

irrlicht

MailDroid on Android
Search Subject “issue” online (Subject from missing mails from trouble ticket system) , not in local phonecache.
Logs from Kopanogateway shows the discoveries, MailDroid can’t open them. eM Client similar problem, Thunderbird no problems,
Searcherror:

Details

<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><report><board>MT6582</board><brand>Lenovo</brand><details>Create messages list screen.</details><domain>wvss-mannheim.de</domain><locale>Deutsch (Deutschland)</locale><model>Lenovo A7600-F</model><package_name>com.maildroid</package_name><package_version>4.91</package_version><protocol>imap</protocol><stack>javax.mail.MessagingException: Failed to load IMAP envelope
at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1238)
at com.sun.mail.imap.IMAPMessage.getSubject(IMAPMessage.java:345)
at com.maildroid.second.ae.a(SessionUtils.java:48)
at com.maildroid.service.o.c(SessionControllerDispatcher.java:107)
at com.maildroid.service.o.a(SessionControllerDispatcher.java:40)
at com.maildroid.service.n.a(SessionController.java:100)
at com.maildroid.service.p.b(SessionControllerWithState.java:99)
at com.maildroid.service.p.a(SessionControllerWithState.java:52)
at com.maildroid.service.m.a(RemoteMailService.java:139)
at com.maildroid.m.e.a(MailService.java:43)
at com.maildroid.bk.c.a(TasksExecutor.java:41)
at com.maildroid.bk.d$1.run(TasksExecutorThread.java:18)
at com.flipdog.commons.u.a$1.run(ThreadUtils.java:46)
at java.lang.Thread.run(Thread.java:761)
</stack><uuid>6793fec8-3f71-4e6c-9591-a8ba47374707</uuid><version_incremental>18c2428f1f</version_incremental><version_release>7.1.2</version_release><version_sdk>25</version_sdk></report>

irrlicht

I build a second kopano-server on a different machine. Mails from t-online are invisiible in different imap-clients, thunderbird is ok, eM Client not.

Could somebody test this please?

fbartels

So on the second server the behaviour is different?

Will check this out sometime in the beginning of next year.

irrlicht

Happy new year.

@fbartels said in 8.7.80.307 missing mails in different imap-clients:

So on the second server the behaviour is different?

No, same behaviour. T-online mails are hidden on MailDroid and eM-Client. Thunderbird ist ok.

irrlicht

Hi,
same problems with nightly kopano-gateway 8.7.80.402.1c7f107ba-0+44.1 on the testserver, it think it’s a problem with kopano-gateway.
Comment from another forum about the javaerror: “server seems to be broken”.

greets from germany

by the way, upgrade to 8.7.80.402 php-mapi-error:

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/mapi.so' - /usr/lib/php/20151012/mapi.so: undefined symbol: _Z25zif_mapi_freebusy_openmsgP18_zend_execute_dataP12_zval_struct in Unknown on line 0
[Wed Jan 02 19:31:21.077162 2019] [mpm_prefork:notice] [pid 12493] AH00163: Apache/2.4.25 (Debian) OpenSSL/1.0.2q configured -- resuming normal operations

irrlicht

php-mapi-error ist fixed with 8.7.80.424, t-online-mails still invisible in MailDroid and eM Client.

Does anybody have the same problems?

fbartels

@irrlicht said in 8.7.80."307-424..." missing mails in different imap-clients:

Worked before with old Zarafa

I did not see this bit before. Which Zarafa version are you speaking about?

We were able to reproduce the email not showing up in emclient with the t-online mail, but not with the mail commands you posted.

irrlicht

Hi Felix and thank you verry much for thist post,

I think it was last Zarafa-Version: 7.2.6.10 on Debian 8 and the mail commands perhaps only work from this one Server, old postfix 2. This machine runs our trouble ticket system. I will have a try from another one.

@fbartels
Edith: It’s strange, from the cli of the trouble ticket machine with postfix 2.11.3-1+deb8u2 without content-type is invisible in eM Client and in MailDroid, with content-type is ok, from other hosts all are ok, with or without.

Shellscript 1:
mail -s “emShit”"$1" user@example.com <<< testtext

Shellscript 2:
mail -s “emShit”"$1" -a “Content-Type: text/plain; charset=utf-8” user@example.com <<< testtext

irrlicht

Strange, in thunderbird are all missing mails from eM-Client visible, but a few others have no topics and no sender. Can’t reproduce them, a few of the missing parts are available a few hours later. In other thunderbird versions are only senders missing.

irrlicht

Are we the only ones with this imap problem? Problem exists with stable 8.6.9 too.

fbartels

for reference, the ticket to follow for this would be https://jira.kopano.io/browse/KC-1375. In our tests we also saw that this occurs with 8.6 as well.

I find it interesting that during the list command the message is shown, but then not picked up by the imap client later on.

Currently busy with a customer escalation, will later spent some more time on this.

irrlicht

I think the problem is fixed in 8.7.0 and in nightly 8.7.80.529

New mails from problematic servers are shown, old ones not.

kopano-dbadm kc-1375 in old installation throws errors

kopano-dbadm kc-1375
Tue Jan 29 20:18:22 2019: [notice ] kc1375: purging problematic-looking IMAP envelopes...
Tue Jan 29 20:18:25 2019: [error  ] 0x55b9bbb977d0: SQL Failed: BIGINT UNSIGNED value is out of range in '(`kopano`.`p`.`tag` - 34049)', Query: "DELETE p FROM properties AS p INNER JOIN names AS n ON p.tag-34049=n.id WHERE n.guid=0x08F1F5003F8EC746AF725E201C2349E7 AND n.nameid=1 AND p.type=30 AND p.val_string LIKE '% (((%))) %'"
Tue Jan 29 20:18:25 2019: [error  ] KDatabase::I_Update() query failed: "BIGINT UNSIGNED value is out of range in '(`kopano`.`p`.`tag` - 34049)'", query: DELETE p FROM properties AS p INNER JOIN names AS n ON p.tag-34049=n.id WHERE n.guid=0x08F1F5003F8EC746AF725E201C2349E7 AND n.nameid=1 AND p.type=30 AND p.val_string LIKE '% (((%))) %'
Tue Jan 29 20:18:25 2019: [error  ] dbadm: action failed: disk error (80000007)

on a new testserver it works

kopano-dbadm kc-1375
Tue Jan 29 20:24:38 2019: [notice ] kc1375: purging problematic-looking IMAP envelopes...
Tue Jan 29 20:24:40 2019: [notice ] kc1375: deleted 121 rows.

@fbartels : “purge” means it deletes the mails in cache or in database?

fbartels

as the man page says:

Deletes all cached IMAP Envelopes that show signs of having address lists with excess parentheses.

jasgggit

Hi everyone,

I’m testing this implementation using the UCS Univention, I’m on the latest builds from them and also using a Trial key, and no updates are available, but :

I’m facing this issue using the mobile app SPARK on an iPhone.

I’m using also the POSTBOX client on a Mac, but on this client, there are no issues.

I tried to run ‘kopano-dbadm kc-1375’ on the shell and got:

root@k4ucs:~# kopano-dbadm kc-1375
Thu Apr 18 16:39:03 2019: [error ] dbadm: unknown action “kc-1375”

Any suggestions?

Regards.
JG

fbartels

@jasgggit said in 8.7.80."307-424..." missing mails in different imap-clients:

I’m facing this issue using the mobile app SPARK on an iPhone.
I’m using also the POSTBOX client on a Mac, but on this client, there are no issues

As you could see already above, the effect did only show up on certain clients. As indicated in the linked ticket you would need to run 8.7.0 to have the fixed version.

jasgggit

@fbartels said in 8.7.80."307-424..." missing mails in different imap-clients:

As you could see already above, the effect did only show up on certain clients. As indicated in the linked ticket you would need to run 8.7.0 to have the fixed version.

AFAIK 8.7 is not yet released, and also its not yet on the UCS app platform too, as I’m running on UCS univention.
Thanks anyway for the reply.
Regards.
JG