working with huge mailboxes is close to impossible

daudo

@fbartels said in working with huge mailboxes is close to impossible:

Hi @daudo ,

@daudo said in working with huge mailboxes is close to impossible:

The office@… account is quite huge (about 20GB in mail) .

large accounts should generally be no problem. Sure the initial sync may take a while, but the again you hopefully only have to do it once.

@daudo said in working with huge mailboxes is close to impossible:

the initial synchronization takes hours (vs. seconds when using the old MAPI client).

You seem to confuse KOE with the old Zarafa mapi client. While the old zarafa client had an online mode that basically worked as a terminal for whatever was stored on the server, KOE/activesync requires all data to be downloaded to the local client first. In fact Zarafa also had a cached/offline mode and while I never measured it, the sync process with koe feels faster.

@daudo said in working with huge mailboxes is close to impossible:

Additionally, the server load is huge during this initial synchronization (vs. minimal with the old MAPI client).

It seems your should revisit the tuning parameters of kopano/mysql/your webserver. https://kb.kopano.io/display/WIKI/Tuning provides a good start, some more in depth guides can be obtained from our support.

Thanks for your input. I’m very aware that the KOE and the old MAPI client are an entirely different approach to the same usecase.

Yes, database tuning may help a little, but I doubt that the time needed to synchronize will get significantly down from the current hours to a more decent value (like minutes).

Part of our problem is probably that we set up the front desks simultaneously, causing the same server load 4 times.

From a technical POV I understand why that is happening, from an user’s perspective this is more difficult to understand (not blaming Zarafa/Kopano for that, no worries). And user experience is pivot as we all know.

So far, the best idea to at least remove parts of the synchronization overhead seems to be to use IMAP instead, which again comes with the disadvantage that it lacks calendaring and tasks. So this might be usable for the back office people, but not for the front desks, they must have access to the calendar & tasks.

Thanks again

Udo

Sebastian

Would a smaller sync window for the emails be an option? E.g. synchronizing only 2 month of emails instead of the full 20 GB of data?

daudo

@sebastian

yes, that would be way better, but if I see it correctly, the only option is “1 month” vs. “all” in the ActiveSync account settings.

And 1 month is definitively not enough.

mcostan

Perhaps if that’s not available to choose from in Outlook, it could be a setting somewhere on the backend in the config in z-push where the users can force a particular sync time? I.e. in Outlook the user selects “Everything” but on the back end if it limited say for a specific user and a particular client, to a specific interval?

externa1

@daudo said in working with huge mailboxes is close to impossible:

So far, the best idea to at least remove parts of the synchronization overhead seems to be to use IMAP instead, which again comes with the disadvantage that it lacks calendaring and tasks. So this might be usable for the back office people, but not for the front desks, they must have access to the calendar & tasks.

Hi Udo,

Yes but the front desk also may be able to use imap for the shared mails, and can use the calendar and tasks over KOE Active Sync

rg
Christian

Gerald

Yes but the front desk also may be able to use imap for the shared mails, and can use the calendar and tasks over KOE Active Sync

AFAIK Outlook 2016 can either sync all of Contacts/Calendar/Mail or nothing. How do you use ActiveSync in Outlook 2016 to sync only the Calendar as you have suggested?

PS: Also, IMAP in Outlook 2016 will keep a local copy of the entire mail database. So if you have 100GB Mails in the shared account every client will need the necessary free disk space for that.

externa1

@gerald

Hi, with Kopano KOE (Kopano Outloook Extension) you can select only Calendar from a shared mailbox

0_1503404644905_5fc62653-683f-4621-97f9-ba1968d0a3bd-grafik.png

Also on IMAP account you can select 1,3,6,12 or 24 month of offline mail download or all

0_1503404600793_20ccba3a-3e23-4c84-812d-d9a0e8934638-grafik.png

rg
Christian

isol

Well the limitation that you need the disk space applies to ActiveSync and IMAP likewise (although with the latter you can limit the sync amount in detail, which we have done otherwise outlook spends too much time syncing IMAP accounts with many subfolders). The fact that all mails are stored locally means you need to treat large mailboxes carefully and plan migration of such accounts with disk space, performance, and sync time in mind.

After migrating a customer with many large mailboxes, I strongly suggest that if you want to use Kopano with Outlook and ActiveSync, then limit your Mailboxes to around 10GB and move the rest to an archival solution. In that way you can keep mailbox handling manageable also for the future - if the mailbox keeps growing, resyncing after e.g. reinstalling the client computer will take longer and longer.

Coffee_is_life

Hello all,

in my config.php from z-push there are some lines:

  154      // Global limitation of items to be synchronized
   155      // The mobile can define a sync back period for calendar and email items
   156      // For large stores with many items the time period could be limited to a max value
   157      // If the mobile transmits a wider time period, the defined max value is used
   158      // Applicable values:
   159      //   SYNC_FILTERTYPE_ALL (default, no limitation)
   160      //   SYNC_FILTERTYPE_1DAY, SYNC_FILTERTYPE_3DAYS, SYNC_FILTERTYPE_1WEEK, SYNC_FILTERTYPE_2WEEKS,
   161      //   SYNC_FILTERTYPE_1MONTH, SYNC_FILTERTYPE_3MONTHS, SYNC_FILTERTYPE_6MONTHS
   162      define('SYNC_FILTERTIME_MAX', SYNC_FILTERTYPE_6MONTHS);

location on centos: /usr/share/z-push/config.php

you can define the sync-filter serverside.

Coffee_is_life

mcostan

Thanks, I must have missed this. Is there any way to configure it by username? I.e. you could have the case where a big mailbox is only for a particular user and the others aren’t therefore they want the entire history.

Sebastian

@coffee_is_life said in working with huge mailboxes is close to impossible:

location on centos: /usr/share/z-push/config.php

If you install from packages you should always edit /etc/z-push/z-push.conf.php
(at the end it’s the same file, but still).

Cheers,
Sebastian