Lack of documentation on how to use kopano-backup - especially regarding the purge function



  • I’m trying to understand how kopano-backup actually works… The man page does unfortunately not go into a lot of detail and there so far has been little discussion on this topic in the forum. I tried to read some comments on bugs discussing development of kopano-backup to get some understanding of what is does.

    For example, the man page says regarding --deletes YESNO
    "The default when backing up is to maintain data which has been soft- or hard-deleted."

    So I would expect that when backing up with “–deletes no” that it would remove deleted information from the backup directory. It does not do this however… The basic behavior of kopano-backup seems to be “never delete any item and just always add to it”… Which is of course not so perfect because if you have a 150MB mail in your inbox and then you put in in /stuff/project_a/ a few days later your backup now contains the mail twice and twice the storage space is needed!

    From what I found out, the only way to get rid of these duplicates in the backup seems to be the purge option? This is however not accepted as an option while doing the backup! It seems to be a separate process to be run later?

    So how do I start --purge? There is absolutely no documentation on this, so i started trying stuff:

    • I was only able to start it for the directories of individual backed-up users, where it seems to do something (kopano-backup --purge 7 /mnt/backup-dir/user/)
    • When I start it for the entire backup (kopano-backup --purge 7 /mnt/backup-dir/) it seems to do nothing at all
    • I could write a script that iterates through the users in /mnt/backup-dir/ but even that would not make sure that stuff that is deleted is gone, because if a user was deleted in kopano this does not seem to make kopano-backup remove the backup for this user at all - with or without purge…

    kopano-backup needs more documentation, especially in the “what does it actually do?” area, the “purge” command needs to be explained, and a “some basic examples of how to use it” somewhere either in the man page or in the kopano administrator manual would be useful… And some hints like “if you delete a user in kopano, you will manually need to remove the users backup because kopano-backup will not do so on its own”.

    Regarding examples:
    Currently the administrators guide recommends the use of filesystem snapshots in the target directory… Which is fine but since we have snapshot anyway the policy of “never delete any mail from the backup ever” does not seem to make sense - that’s what the snapshots are for…

    Perhaps there should be two ways with examples how to use kopano-backup

    1. with “delete stuff that was deleted” for filesystem destinations that provide snapshots
    2. one with “just keep everything forever” for filesystem destinations that do not provide snapshots?

    In case someone is interested regarding how I’m using it (posting this because there are so few examples):
    I’m currently using crontab to run

    kopano-backup --skip-deleted --worker-processes 1 --output-dir /mnt/btrfsBackup/kopano-backup/
    

    3 days a week and afterwards rotate the btrfs snapshot using my own custom magic :relaxed: (which keeps and removes snapshots to satisfy my retention goals).

    I had crashes with more than 1 worker-process but that was quite some time ago and may very well have been fixed. But incremental backups are fast anyway. This seems to work well, except for the missing purge of no longer existing items…



  • I’m still wondering how kopano-backup is actually supposed to work. I’ve just looked at a couple of backup folders which were created during the past months by a weekly run of

    /usr/sbin/kopano-backup --deletes no --skip-deleted --worker-processes 1 --output-dir /mnt/path/kopano-backup-dir/
    

    The resulting backup directory contains 37GB of files.

    When I make a completely new backup into a fresh directory, the backup is only 29GB big.
    Which made me wonder why the backup that grew over time is bigger even though I specified “–deletes no”?

    So I ran

     /usr/sbin/kopano-backup --purge 1  /mnt/btrfsBackup/test-old/
     /usr/sbin/kopano-backup --purge 0  /mnt/btrfsBackup/test-old/
    

    which did not change anything. The “legacy” backup is always 8GB bigger than the freshly created backup and there seems to be no way to get those two back in sync except deleting the entire thing.

    The new kopano-backup is really weird :confused:


  • Kopano

    Hi @gerald ,

    maybe you should run your commands with -l debug to see what the program is executing.

    @gerald said in Lack of documentation on how to use kopano-backup - especially regarding the purge function:

    if a user was deleted in kopano this does not seem to make kopano-backup remove the backup for this user at all - with or without purge…

    from my experience admins want to keep the last backup set of a deleted user. just imagine the following:

    • you accidentally delete a user
    • run kopano-backup
    • and only then realise that you accidentally deleted the user and want to restore him
      -> you could not do this if kopano-backup would purge backups of non existing users.

Log in to reply
 

Looks like your connection to Kopano Community Forum was lost, please wait while we try to reconnect.