working with huge mailboxes is close to impossible


  • Kopano

    Hi @daudo ,

    @daudo said in working with huge mailboxes is close to impossible:

    The office@… account is quite huge (about 20GB in mail) .

    large accounts should generally be no problem. Sure the initial sync may take a while, but the again you hopefully only have to do it once.

    @daudo said in working with huge mailboxes is close to impossible:

    the initial synchronization takes hours (vs. seconds when using the old MAPI client).

    You seem to confuse KOE with the old Zarafa mapi client. While the old zarafa client had an online mode that basically worked as a terminal for whatever was stored on the server, KOE/activesync requires all data to be downloaded to the local client first. In fact Zarafa also had a cached/offline mode and while I never measured it, the sync process with koe feels faster.

    @daudo said in working with huge mailboxes is close to impossible:

    Additionally, the server load is huge during this initial synchronization (vs. minimal with the old MAPI client).

    It seems your should revisit the tuning parameters of kopano/mysql/your webserver. https://kb.kopano.io/display/WIKI/Tuning provides a good start, some more in depth guides can be obtained from our support.



  • @fbartels said in working with huge mailboxes is close to impossible:

    […] the old zarafa client had an online mode that basically worked as a terminal for whatever was stored on the server, KOE/activesync requires all data to be downloaded to the local client first.

    This was absolutely great about the old client. It was possible to have a 100GB shared Mailbox for all users that was easily accessible within the office (remote access was not so great) without the need for every single client machine to have a 100GB local copy of the shared mails. Now with Kopano/Activesync every single client PC will store a local copy of those 100GB, which will then also increase the backup storage needed for each of those clients from 30GB (normal Windows Installation) to 130GB (Win10+Mails).

    And now if there are synchronsation problems you have to trigger a full resync of the 100GB. Not looking forward to doing this :-)
    Summary: everything was better in the old days :cry:
    (ok, not everything. Mobile use in Outlook with activesync is soooo much better than the old zarafa cached mode)



  • @fbartels said in working with huge mailboxes is close to impossible:

    Hi @daudo ,

    @daudo said in working with huge mailboxes is close to impossible:

    The office@… account is quite huge (about 20GB in mail) .

    large accounts should generally be no problem. Sure the initial sync may take a while, but the again you hopefully only have to do it once.

    @daudo said in working with huge mailboxes is close to impossible:

    the initial synchronization takes hours (vs. seconds when using the old MAPI client).

    You seem to confuse KOE with the old Zarafa mapi client. While the old zarafa client had an online mode that basically worked as a terminal for whatever was stored on the server, KOE/activesync requires all data to be downloaded to the local client first. In fact Zarafa also had a cached/offline mode and while I never measured it, the sync process with koe feels faster.

    @daudo said in working with huge mailboxes is close to impossible:

    Additionally, the server load is huge during this initial synchronization (vs. minimal with the old MAPI client).

    It seems your should revisit the tuning parameters of kopano/mysql/your webserver. https://kb.kopano.io/display/WIKI/Tuning provides a good start, some more in depth guides can be obtained from our support.

    Thanks for your input. I’m very aware that the KOE and the old MAPI client are an entirely different approach to the same usecase.

    Yes, database tuning may help a little, but I doubt that the time needed to synchronize will get significantly down from the current hours to a more decent value (like minutes).

    Part of our problem is probably that we set up the front desks simultaneously, causing the same server load 4 times.

    From a technical POV I understand why that is happening, from an user’s perspective this is more difficult to understand (not blaming Zarafa/Kopano for that, no worries). And user experience is pivot as we all know.

    So far, the best idea to at least remove parts of the synchronization overhead seems to be to use IMAP instead, which again comes with the disadvantage that it lacks calendaring and tasks. So this might be usable for the back office people, but not for the front desks, they must have access to the calendar & tasks.

    Thanks again

    Udo


  • Kopano

    Would a smaller sync window for the emails be an option? E.g. synchronizing only 2 month of emails instead of the full 20 GB of data?



  • @sebastian

    yes, that would be way better, but if I see it correctly, the only option is “1 month” vs. “all” in the ActiveSync account settings.

    And 1 month is definitively not enough.


  • Translator

    Perhaps if that’s not available to choose from in Outlook, it could be a setting somewhere on the backend in the config in z-push where the users can force a particular sync time? I.e. in Outlook the user selects “Everything” but on the back end if it limited say for a specific user and a particular client, to a specific interval?



  • @daudo said in working with huge mailboxes is close to impossible:

    So far, the best idea to at least remove parts of the synchronization overhead seems to be to use IMAP instead, which again comes with the disadvantage that it lacks calendaring and tasks. So this might be usable for the back office people, but not for the front desks, they must have access to the calendar & tasks.

    Hi Udo,

    Yes but the front desk also may be able to use imap for the shared mails, and can use the calendar and tasks over KOE Active Sync

    rg
    Christian



  • Yes but the front desk also may be able to use imap for the shared mails, and can use the calendar and tasks over KOE Active Sync

    AFAIK Outlook 2016 can either sync all of Contacts/Calendar/Mail or nothing. How do you use ActiveSync in Outlook 2016 to sync only the Calendar as you have suggested?

    PS: Also, IMAP in Outlook 2016 will keep a local copy of the entire mail database. So if you have 100GB Mails in the shared account every client will need the necessary free disk space for that.



  • @gerald

    Hi, with Kopano KOE (Kopano Outloook Extension) you can select only Calendar from a shared mailbox

    0_1503404644905_5fc62653-683f-4621-97f9-ba1968d0a3bd-grafik.png

    Also on IMAP account you can select 1,3,6,12 or 24 month of offline mail download or all

    0_1503404600793_20ccba3a-3e23-4c84-812d-d9a0e8934638-grafik.png

    rg
    Christian



  • Well the limitation that you need the disk space applies to ActiveSync and IMAP likewise (although with the latter you can limit the sync amount in detail, which we have done otherwise outlook spends too much time syncing IMAP accounts with many subfolders). The fact that all mails are stored locally means you need to treat large mailboxes carefully and plan migration of such accounts with disk space, performance, and sync time in mind.

    After migrating a customer with many large mailboxes, I strongly suggest that if you want to use Kopano with Outlook and ActiveSync, then limit your Mailboxes to around 10GB and move the rest to an archival solution. In that way you can keep mailbox handling manageable also for the future - if the mailbox keeps growing, resyncing after e.g. reinstalling the client computer will take longer and longer.



  • Hello all,

    in my config.php from z-push there are some lines:

      154      // Global limitation of items to be synchronized
       155      // The mobile can define a sync back period for calendar and email items
       156      // For large stores with many items the time period could be limited to a max value
       157      // If the mobile transmits a wider time period, the defined max value is used
       158      // Applicable values:
       159      //   SYNC_FILTERTYPE_ALL (default, no limitation)
       160      //   SYNC_FILTERTYPE_1DAY, SYNC_FILTERTYPE_3DAYS, SYNC_FILTERTYPE_1WEEK, SYNC_FILTERTYPE_2WEEKS,
       161      //   SYNC_FILTERTYPE_1MONTH, SYNC_FILTERTYPE_3MONTHS, SYNC_FILTERTYPE_6MONTHS
       162      define('SYNC_FILTERTIME_MAX', SYNC_FILTERTYPE_6MONTHS);
    

    location on centos: /usr/share/z-push/config.php

    you can define the sync-filter serverside.

    Coffee_is_life


  • Translator

    Thanks, I must have missed this. Is there any way to configure it by username? I.e. you could have the case where a big mailbox is only for a particular user and the others aren’t therefore they want the entire history.


  • Kopano

    @coffee_is_life said in working with huge mailboxes is close to impossible:

    location on centos: /usr/share/z-push/config.php

    If you install from packages you should always edit /etc/z-push/z-push.conf.php
    (at the end it’s the same file, but still).

    Cheers,
    Sebastian


Log in to reply
 

Looks like your connection to Kopano Community Forum was lost, please wait while we try to reconnect.