Empty emails on Outlook 2016 & 2019 with Kopano 10

deajan

Thanks for your help Manfred.

I have setup kopano-gateway and connected an outlook profile with an IMAP profile.
So the problematic mails appear both in Z-Push and IMAP sync modes, so I can outrule Z-push.

I’ve resubmitted failed mails from the antispam, they are sent as .msg attachment, and appear correct in Outlook.
I’ve then connected directly into the antispam via ssh, and resubmitted two mails for delivery with /usr/sbin/sendmail.postfix -t < /var/spool/Mailscanner/20200831/nospam/<mymailID>. Those mails are raw format.

The mails show in outlook without any problem, using z-push or imap.

I’ve also sent two test mails from another mail server… One came as empty message, the other came in ok.

I have no clue where to search now.
I have activated dagent raw logging, but since this is totally random, it will be searching a needle in a haystack.

I’m pretty sure I won’t need the WBXML logs since I outruled Z-Push, right ?

All I can do is leaving dagent raw logging enabled, and wait for the next missing mails, am I right ?

Best regards,
Orsiris.

Manfred

Hi Orsiris,

@deajan said in Empty emails on Outlook 2016 & 2019 with Kopano 10:

I’ve also sent two test mails from another mail server… One came as empty message, the other came in ok.

Well, what was the difference between two emails? Maybe some special chars (e.g. German umlauts ä ö ü) in one? Or the content type (plain vs html)?

I’m pretty sure I won’t need the WBXML logs since I outruled Z-Push, right ?

Yes, if you can rule out Z-Push, then the WBXML log is not necessary.

All I can do is leaving dagent raw logging enabled, and wait for the next missing mails, am I right ?

Unfortunately I also don’t have any other idea. Maybe you could post the same question in the core section of the forum now that Z-Push or KOE seem not to be the ones breaking the emails.

Manfred

deajan

Hello Manfred,

Well, what was the difference between two emails? Maybe some special chars (e.g. German umlauts ä ö ü) in one? Or the content type (plain vs html)?

Both mails are litteraly the same, written by another kopano instance I had earlier. They have the same subject, and contain “test” and “test 2”, the latter being received without problems.

I also suspected some problems with mysql, and increased max_packet_size from 16MB to 128MB, but that shouldn’t be the problem, since my test emails should weight some KBs only.

Anyway, I’ve kept dagent raw logs on, and am currently waiting for the next problematic emails (non as of today). I still wonder why this problem is so random. I’ve looked at dmesg and various other log files just to make sure the system wasn’t faulty too.

Thanks for your help.
A last thing, I would like to “extract” a problematic mail from kopano DB, to check whether it is fully RFC822 compliant. Is there any kopano tool I can export a message as raw ?

Thanks,
Orsiris.

alexthetiger68

I am on the same OS and Kopano Version as deajan and experience the same empty mails in Outlook via ActiveSync (Z-Push).

osrelease               Pretty operating system name                                                    Debian GNU/Linux 10 (buster)
program_name            Program name                                                                    kopano-server
program_version         Program version                                                                 10.0.6
server_guid                                                                                             7636E72C7285493384B2CCC3487C6508

What is interesting, the same empty Mail is not only perfectly visible in Kopano Webapp, but also on another device synced via z-push (iphone) to the same account. I dont have the KOE extensions installed.

I did not see any error messages in the mysql logs.

The kopano-server 10.0.6.406.69083f1d7-0+217.1 seems to have at least two major issues, the lost connections to LDAP and these empty mails. Do you have an ETA when the nightly artefacts will be updated again so we can test more? Or maybe you can point me to the correct repository for kopano-core so I can compile the sources on my own. I only found a repo with tons of forks and am a bit lost as to what is the master repo.

deajan

@alexthetiger68 I don’t experience more of those empty messages for the moment (strange enough, it’s random).

Maybe it’s related to the following mysql settings I tuned

[mysqld]
max_allowed_packet=128M
innodb_log_file_size = 128M # Fix kopano-server: SQL [00000088] info: MySQL server has gone away. Reconnecting, see https://jira.kopano.io/browse/KC-1053

The second setting needs you to remove your ib_log files with

systemctl stop mariadb
mv /var/lib/mysql/ib_log* /root
# make changes in /etc/mysql/mariadb.conf.d/50-server.cnf
systemctl restart mariadb

Can you try these and report back if it worked for you ?
Thanks.

alexthetiger68

@deajan I’ve made the changes to the mysqld and will monitor this issue.

I’ve seen a Warning in the mysqld log, but right now I do not consider this to be a major problem.

Sep 08 10:59:21 fizzi2 systemd[1]: Starting MariaDB 10.3.23 database server…
Sep 08 10:59:21 fizzi2 mysqld[23436]: 2020-09-08 10:59:21 0 [Note] /usr/sbin/mysqld (mysqld 10.3.23-MariaDB-0+deb10u1) starting as process 23436 …
Sep 08 10:59:21 fizzi2 mysqld[23436]: 2020-09-08 10:59:21 0 [Warning] Could not increase number of max_open_files to more than 16364 (request: 32184)
Sep 08 10:59:22 fizzi2 systemd[1]: Started MariaDB 10.3.23 database server.

deajan

@alexthetiger68 said in Empty emails on Outlook 2016 & 2019 with Kopano 10:

Could not increase number of max_open_files to more than 16364

How big is your setup ? (ie how many users / gigabytes of data)
Is your MySQL shared ?

Unless your mysql is huge, you should check why it has open more than 16k files.

If your usecase requires to, you should definitly tune mysql, see https://haydenjames.io/could-not-increase-number-of-max_open_files-to-more-than/

alexthetiger68

@deajan I’m currently running Kopano in a container on Proxmox. The database has been converted from Zarafa to Kopano a long time ago and is currently 46GB in size. I’ve made the switch from VM to container about two years ago and the system has always been running smooth, until I upgraded to from from 10.0.5.5.8827066e2-0+106.1 to core-10.0.6.406.e263d46-Debian_10-amd64 .

deajan

Honestly I run my personal Kopano with less than 25G, so I can’t tell how much connections you should have open at the same time.
But nevertheless, you should have a look at why you need more than 16K files open before tuning this setting. Maybe SHOW STATUS WHERE variable_name = 'Max_used_connections'; could help here.

Manfred

Hi Orsiris,

@deajan said in Empty emails on Outlook 2016 & 2019 with Kopano 10:

A last thing, I would like to “extract” a problematic mail from kopano DB, to check whether it is fully RFC822 compliant. Is there any kopano tool I can export a message as raw ?

In webapp right click on the email and select “Export as” -> EML file(s).

Manfred

deajan

Thanks ;) I’ll check this ASAP.