kopano-backup: Not finished after 14 days?

Allyfied

Hi,
for some weeks now, the kopano-backup will not finish it’s task. After 2 weeks i killed the backup.
This is, what i see, when i start the kopano-backup manually without using the crontab:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/kopano/log.py", line 103, in log_exc
    try: yield
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 366, in update
    self.commit()
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 400, in commit
    index_db[sourcekey.encode('ascii')] = idx
  File "/usr/lib/python3.5/contextlib.py", line 162, in __exit__
    self.thing.close()
  File "/usr/lib/python3/dist-packages/bsddb3/__init__.py", line 265, in close
    v = _DeadlockWrap(self.db.close)
  File "/usr/lib/python3/dist-packages/bsddb3/dbutils.py", line 67, in DeadlockWrap
    return function(*_args, **_kwargs)
bsddb3.db.DBRunRecoveryError: (-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB0060 PANIC: fatal region error detected; run recovery')

2021-08-21 08:34:02,428 - backup0 - ERROR - Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 398, in commit
    item_db[sourcekey.encode('ascii')] = data
  File "/usr/lib/python3/dist-packages/bsddb3/__init__.py", line 248, in __setitem__
    _DeadlockWrap(wrapF)  # self.db[key] = value
  File "/usr/lib/python3/dist-packages/bsddb3/dbutils.py", line 67, in DeadlockWrap
    return function(*_args, **_kwargs)
  File "/usr/lib/python3/dist-packages/bsddb3/__init__.py", line 247, in wrapF
    self.db[key] = value
bsddb3.db.DBRunRecoveryError: (-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB0061 PANIC: Resource deadlock avoided')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/kopano/log.py", line 103, in log_exc
    try: yield
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 220, in main
    self.backup_hierarchy(path, stats, options, store, user, server, config)
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 287, in backup_hierarchy
    self.backup_folder(data_path, data_path, folder, subtree, config, options, stats, store, user, server)
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 325, in backup_folder
    importer.commit()
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 400, in commit
    index_db[sourcekey.encode('ascii')] = idx
  File "/usr/lib/python3.5/contextlib.py", line 162, in __exit__
    self.thing.close()
  File "/usr/lib/python3/dist-packages/bsddb3/__init__.py", line 265, in close
    v = _DeadlockWrap(self.db.close)
  File "/usr/lib/python3/dist-packages/bsddb3/dbutils.py", line 67, in DeadlockWrap
    return function(*_args, **_kwargs)
bsddb3.db.DBRunRecoveryError: (-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB0060 PANIC: fatal region error detected; run recovery')

2021-08-21 08:34:04,134 - backup - WARNING - Sync state does not exist on server (anymore); requesting new one
2021-08-21 08:34:08,323 - backup0 - ERROR - Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/kopano/log.py", line 103, in log_exc
    try: yield
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 220, in main
    self.backup_hierarchy(path, stats, options, store, user, server, config)
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 287, in backup_hierarchy
    self.backup_folder(data_path, data_path, folder, subtree, config, options, stats, store, user, server)
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 325, in backup_folder
    importer.commit()
  File "/usr/lib/python3/dist-packages/kopano_backup/__init__.py", line 398, in commit
    item_db[sourcekey.encode('ascii')] = data
  File "/usr/lib/python3/dist-packages/bsddb3/__init__.py", line 248, in __setitem__
    _DeadlockWrap(wrapF)  # self.db[key] = value
  File "/usr/lib/python3/dist-packages/bsddb3/dbutils.py", line 67, in DeadlockWrap
    return function(*_args, **_kwargs)
  File "/usr/lib/python3/dist-packages/bsddb3/__init__.py", line 247, in wrapF
    self.db[key] = value
bsddb3.db.DBPageNotFoundError: (-30986, 'BDB0075 DB_PAGE_NOTFOUND: Requested page not found')

2021-08-21 08:34:46,430 - backup - WARNING - Sync state does not exist on server (anymore); requesting new one
2021-08-21 08:34:46,539 - backup - WARNING - Sync state does not exist on server (anymore); requesting new one

Is there anything i could do about this?
For the Sync state i already reindexed kopano-search.
But i would really love the backup to be finished in 24 hours. All user stores together are around 250 gigabytes. That should last not so long on a gigbit network.

kopano-server is version 8.7.1.0-1
kopano-webapp: 3.5.14.2539-2
z-push: 2.6.2-1
Server is UCS 4.4-8 errata 992

Best regards
Andre

@allyfied Your software is quite old, try to update to Kopano 8.7.22. With 8.7.22 sometimes I see similar messages, but kopano-backup worked. Two weeks ago I need to restore some mail boxes for a customer.

Allyfied

@walterhof
i don’t why, but the backup job was finished after 4 hours. If it remains so, everything is fine again.
Best regards
Andre