Navigation

    Kopano
    • Register
    • Login
    • Search
    • Categories
    • Get Official Kopano Support
    • Recent
    Statement regarding the closure of the Kopano community forum and the end of the community edition

    kopano-search: Unable to create (new) sync-state

    Kopano Groupware Core
    3
    4
    498
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • umgfoin
      umgfoin last edited by umgfoin

      Hello community,
      I came over this while looking into the kopano-server[]: setSyncStatus(): collision:

      /var/log/kopano/search.log:

      2018-10-23 02:56:00,972 - search - WARNING - Sync state does not exist on server (anymore); requesting new one
      2018-10-23 02:56:00,979 - search - ERROR - Traceback (most recent call last):
        File "/usr/lib/python2.6/site-packages/kopano-8.7.80-py2.6.egg/kopano/log.py", line 103, in log_exc
          try: yield
        File "/usr/lib/python2.6/site-packages/kopano_search-8.7.80-py2.6.egg/kopano_search/__init__.py", line 393, in incremental_sync
          new_state = self.server.sync(importer, self.state, log=self.log)
        File "/usr/lib/python2.6/site-packages/kopano-8.7.80-py2.6.egg/kopano/server.py", line 774, in sync
          return _ics.sync(self, self.mapistore, importer, state, max_changes, window=window, begin=begin, end=end, stats=stats)
        File "/usr/lib/python2.6/site-packages/kopano-8.7.80-py2.6.egg/kopano/ics.py", line 257, in sync
          syncid, changeid = struct.unpack('<II', _bdec(state))
      error: unpack requires a string argument of length 8
      

      As we can see, the search-daemon is trying to create a new sync-state in ics.py: sync(server, syncobj, importer, state...)

      In order to do so, syncid and changeid are decoded from the beginning of a binary state-object:

      syncid, changeid = struct.unpack('<II', _bdec(state)) 
      

      The unpack routine here exspects a stringobject of exact length sizeof(2 * int32_t), which regularly fails, if the stateobject-stream contains additional data.

      Thus, in my opinion, the string-decoded binary-stream needs truncation to 8 chars prior unpacking:

      syncid, changeid = struct.unpack('<II', _bdec(state)[:8]) 
      

      Proposed patch here.

      ++umgfoin.

      bmaehr 1 Reply Last reply Reply Quote 0
      • fbartels
        fbartels Kopano last edited by

        Hi @umgfoin ,

        I quickly talked with a developer about this. The reason why you see this error is since the sync-state is now again back to its shorter size, but you index may still contain sync-states that are bigger than the expected length. The general solution here should be to drop you index and reindex your stores (all sync states should be then again the expected length).

        The next days he wants to look into python-kopano to give a general error handling of sync states are too long and then also enhance logging so that the admin can better identify such objects.

        Regards Felix

        Resources:
        https://kopano.com/blog/how-to-get-kopano/
        https://documentation.kopano.io/
        https://kb.kopano.io/

        Support overview:
        https://kopano.com/support/

        1 Reply Last reply Reply Quote 0
        • bmaehr
          bmaehr @umgfoin last edited by

          @umgfoin I had the same error, but adding [:8] didn’t fix the error. Any other ideas how to fix the size?

          bmaehr 1 Reply Last reply Reply Quote 0
          • bmaehr
            bmaehr @bmaehr last edited by

            After a long time I was able to fix it with a simple command:

            rm -r /var/lib/kopano/search
            
            1 Reply Last reply Reply Quote 0
            • First post
              Last post