z-push 2.4.5 sync issue - 0xFFFFFFFF8004010F

Manfred

@vincentvana bumping jengelh’s question. How did you remove that attachment?

weini

I´m also still facing the same issue. I´m facing the issue for mail folders with many (starts with a few 1.000 up to 15.000) entries.

It might be interesting, that the issue only happens when I sync to my mobile (Samsung S8). Sync is limited to last 30 days for emails there. I´m syncing the same inbox via EAS also to Outlook without any time restriction, but that never had an issue.

What I did in the meantime, and what DIDN´T solve the issue:

backup user store with kopano-backup and restore it
play around with MariaDB parameters, esp. max_allowed_packet
rebuild tproperties completely with the script provided in the kopano wiki
use kopano-dbadm to (potentially) cleanup db schema

Only sucessful workaround was to move from MariaDB 10.3 to MySQL. The issue was gone there immediately. I used the Docker image to do this. However, having MySQL 8.0 and MariaDB 10.3 running at the same time (as I need MariaDB for some other stuff) is causing a high load on my server, so I want to avoid this scenario in the mid term.

Hint: If you want to go for a similar approach, do yourself a favor and don´t use MySQL 8.0, but version 5.7 instead. You can move tables / schemas between MariaDB 10.3 (and also 10.4) and MySQL 5.7 very easily via InnoDB tablespace import/export. However, if you use MySQL 8.0, this is a one way trip. Going back from MySQL 8.0 to MariaDB requires a full logical export / import which is quite time consuming.

weini

@rolek said in z-push 2.4.5 sync issue - 0xFFFFFFFF8004010F:

I can report that the patch from https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=939751;filename=fix-zpush-sync-error.patch;msg=15 fixes the problem for us.

@fbartels: Did you integrate that patch into your codebase?

robgnu

I was also facing the same issue. I got a big log flooding with the same message. z-push.log was about 10 GBytes after a day.
I solved the issue, but I am not sure, what the real problem was. So I hope, these information will help anyone:

All users are connected to multiple Outlooks 365 via ActiveSync (no KOE)
Some of the users are connected via an Android Phone (Samsung)
The users have many (>4000) calendar entries.
Sync of calendar items between Outlook seems to work, but didn’t sync to the phones. Deleting and recreation of accounts on the phone didn’t solve the issue.
kopano-backup (backup + restore) did NOT solve the issues.
Changing mariadb parameters did NOT solve the issues.
Switching away from MariaDB to Mysql is currently not an option.

There were so many log entrys, that I decided to remove the affected usersaccounts in Outlook and recreated them. After resync the log flood was gone.

Currently working on Debian 10 (Kopano 8.7.85.0.a02e61d66-0+176.1 with z-push 2.5.1+0-0)

bye
Robert.

weini

@robgnu
What exactly do you mean by “remove affected useraccounts in Outlook”?
I delete the account in Outlook and removed it then in z-push. Afterwards, I connected Outlook again.
Unfortunately, this didn´t cure the issue on my side. Did you do any service restarts in between?

fbartels

@weini said in z-push 2.4.5 sync issue - 0xFFFFFFFF8004010F:

@fbartels: Did you integrate that patch into your codebase?

No, since there was never an answer to https://forum.kopano.io/post/15294

weini

@weini said in z-push 2.4.5 sync issue - 0xFFFFFFFF8004010F:

Unfortunately, this didn´t cure the issue on my side. Did you do any service restarts in between?

It seems I was a bit too fast: about 1h after I deleted the Outlook account (and recreated it), the errors are gone and my mobile is syncing the folders that it was formerly refusing to do so.

Fingers crossed…

robgnu

@weini
I removed the Outlook account and recreated the account.

Today I faced some issues again. Now I’m trying to backup/restore the accounts with kopano-backup into a new store. After that I request a resync via z-push-admin to the android devices.

I had luck with one account today. I’m running currently a batch job to recreate a bunch of users. I can tell more tomorrow…

There must be some deeper problems…

Robert.

robgnu

A small update:
After backup users and restoring to a new store the log flood is back:

There are more than 10 entries per second in the logfile (z-push-erro.log):

11/12/2019 06:17:56 [ 1748] [ERROR] [user] StatusException: ExportChangesICS->Config(): Error, exporter not available - code: 12 - file: /usr/share/z-push/backend/kopano/exporter.php:126

The same - more than 10 entries per second in z-push.log:

11/12/2019 06:22:15 [ 9049] [ INFO] [user] cmd='Sync' memory='2.32 MiB/2.00 MiB' time='0.02s' devType='WindowsOutlook' devId='4beabac1b19844dd946eefb2afec7a6c' getUser='user' from='2003:xxxx' idle='0s' version='2.5.1+0-0' method='POST' httpcode='200'

Sometimes this messages are logged multiple times:

11/12/2019 06:37:07 [26162] [ INFO] [user] NoHierarchyCacheAvailableException: Folderid 'Ub8896' is not fully synchronized on the device - code: 0 - file: /usr/share/z-push/lib/core/devicemanager.php:466
11/12/2019 06:37:07 [26162] [ INFO] [user] HandlePing(): unknown collection 'Ub8896', triggering HierarchySync
11/12/2019 06:37:07 [26162] [ INFO] [user] NoHierarchyCacheAvailableException: Folderid 'U70eec' is not fully synchronized on the device - code: 0 - file: /usr/share/z-push/lib/core/devicemanager.php:466
11/12/2019 06:40:11 [31742] [WARN] [user] Not possible to determine class of request. Request did not contain class and apparently there is an issue with the HierarchyCache.

Today I will “go around” and reconfigure the Outlook accounts.

Bye
Robert

fbartels

Hi @robgnu,

in another post you said you we’re using 8.7.7 which would be one of our supported builds. If you have a subscription please reach out to our support so we can have a look at your system.

robgnu

@fbartels
That’s correct. Some of my clients have 8.7.7 with a subscription. Some smaller clients are still using the community builds (installations since 2015 - Zarafa). Switching them to an official build/subscription is in discussion and on it’s way.
All 8.7.7. installations are still on Debian 9, because there are no official Debian 10 builds. :-(

The problem hits all installations with the community build after upgrading Debian 9 to 10 and 8.7.80.307 (stretch) to 8.7.85.0 (buster).

I can give another short update:
After restoring all users to a new store, the Android clients are fine. But some Outlook 365 did a big logflood (see my post above). Today I walked to each computer and recreated all active sync accounts. Currently everything works as before.

Bye
Robert

weini

Brief update from my side as well:

Issue came back in the mobile sync. I dropped the Outlook profile once again and recreated it. Reason is, that Outlook resync had some issues yesterday, so I wanted to give it a 2nd try. Currently, mobile sync is working fine angain and Outlook resync has completed.

From my perspective, the “pattern” looks as follows:

MariaDB as database
EAS profiles for Outlook and also for a mobile device
(possibly) “old” installations with many migration steps in between

weini

This time it worked for 2 days. Now folders do no longer sync on the mobile and the error message is back…
So not a sustainable workaround, at least not for me!

jengelh

The environment variable can be set to KOPANO_CLIENT_LOGLEVEL=0x800006 to get sync messages. That is something introduced in 3ac65333, so for those who feel lucky and in a dev mood…

hrobbie1981

Not sure if it helps, but I do only see this issue as long as my mobile device is set to sync “less than no Limit” days. Specifically on the iPhone in accounts if I set “Mail Days to Sync” to “No Limit” the error does not show up anymore…

Not really what I want, though, as my mailbox is huge…

robgnu

@hrobbie1981
I can not confirm. I see the issues on users where Android Phones (Samsung) and Outlook are connected. The sync limit is set to “All / No Limit”.
Users who using an iPhone doesn’t have any problems as much as I know.

Bye
Robert.

glotzkowsky

Hi. I got the same issue with an android mobile today after upgrading the SQL server VM from Debian 8 to Debian 10. My setup is:
Zarafa 7.2.6-10 (still on Debian 7)
Z-Push 2.2.8
DB: mariadb 10.3.18

For me setting “max_allowed_packet” in my.conf in the [mysqld] Section, resolved the issue immediately.

weini

@glotzkowsky said in z-push 2.4.5 sync issue - 0xFFFFFFFF8004010F:

For me setting “max_allowed_packet” in my.conf in the [mysqld] Section, resolved the issue immediately.

I´d be more than happy, if this would be a sustainable solution…
Unfortunately, others already reported back in this thread, that increasing max_allowed_packet did not cure the issue for them.

To which value did you set “max_allowed_packet”?
Could you please keep us updated, if the issue is gone for you on a permanent base?

glotzkowsky

I set it to 64M, but unfortunately it does not solve the problems for all users. There is one user with > 30000 mails in his inbox. For this user the problem still exists.
Before the update, it was a Mysql 5.5 server with max_allowed_packet set to default (16M).

thctlo

@glotzkowsky 30k emails in the inbox. thats crazy.
It slows down everything…

read : https://support.microsoft.com/en-us/help/2768656/outlook-performance-issues-when-there-are-too-many-items-or-folders-in

MS advices max 5k per folder, so make that guy clean up its mail really, its that simple.