kopano backup extremely slow

tbone

I used to have a script that runs on a daily basis to zarafa-backup all my users to a location on NFS server (2GBit link to that server). The scripts loops processed output of zarafa-admin -l and usually ran from 19h to 22h-23h. On fridays public folders where also dumped.

I have modifed this script to use the kopano commands and the 19h session from last night is still running and its not even half way?!

drwxr-xr-x 3 root root 4096 Nov  7 19:00 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 19:53 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 20:16 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 21:16 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 22:32 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 23:16 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 23:30 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 23:39 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  7 23:53 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 00:48 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 01:17 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 02:40 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 04:08 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 04:15 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 04:28 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 10:11 userxxxxxxxxxxx
drwxr-xr-x 3 root root 4096 Nov  8 10:19 userxxxxxxxxxxx

I understand this new backup tool is working incrementally, can I expect a faster backup next run ? Where does this performance drop come from ? my backup server is idling at peaks of only 2-3 mb/s at best with q write operations where this used to be much higher. Other write tests on this nfs volume indicate performance peaking at 150mb/s on average 97mb/s so its not a storage issue. …

iperf indicated the following so its not a network issue either:
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 2.08 GBytes 1.79 Gbits/sec

These users are only a couple of gigs each, Im curious to see how long the public folder backup is going to take…

Regards,

tbone

After some research Im assuming that this performance issue comes from Berkeley DB not running well on NFS filesystem…

fbartels

Hi @tbone ,

you are not disclosing the version of Kopano Core in this post, but in another you are saying you are still on 8.3.0. We are continuously improving performance of kopano-backup. I’d therefore recommend to either upgrade to the latest stable (if you have a subscription) or the latest nightly and try again.

PS: reading a bit in between the lines it sounds like you are running kopano-backup only for a user at a time. kopano-backup has an option to run backup with multiple threads in parallel, that should also improve runtime.

tbone

Hi @fbartels,

thanks for these tips, I will try them. Perhaps Im doing a user per user thing out of habit from zarafa-backup. I will try but I get the impression its really nfs causing the issue. I will try this. Thank you for feedback!