Unable to keep appointments after foldersync



  • I am using z-push v2.5 with a self-programmed backend, which gets all the necessary data from my database. I try to sync the appointments to an android device and it syncs just fine… but a few seconds / minutes later it sends another request with a FolderSync command and then all appointments (I have exactly one to keep the data transferred simple, but it also happens with >1 appointment) are gone. Even the account is no longer shown eg in google-calendar.

    Here’s the pastebin with the log: https://pastebin.com/96LzMeH6

    The rogue FolderSync starts at line 534, 13/02/2020 15:16:11

    I tried a lot of things up and down and I see no improvement. I am suspecting that my backend somehow mangles the necessary reply, but I just don’t understand what is wrong or how the data would look if it was right.

    I am also suspecting this other post is about more or less the same problem: https://forum.kopano.io/topic/2982/unable-to-sync-contacts

    Thanks for reading,
    BD



  • Not sure why but, at lines 568-570 the SyncKey is received from the client

    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] I   <FolderHierarchy:SyncKey>
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] I    {8e6add07-3b09-4e2b-8809-7ffd9c80955c}1
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] I   </FolderHierarchy:SyncKey>
    
    

    then line 583 reports that there are no Hierarchy changes … but still the SyncKey is changed and sent out. I would expect the received SyncKey to be returned to the client if nothing changed.

    13/02/2020 15:16:11 [ 4161] [ INFO] [zpushy@doen] ExportChangesDiff->InitializeExporter(): Found '0' changes for 'hierarchy'
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] O  <FolderHierarchy:FolderSync>
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] O   <FolderHierarchy:Status>
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] O   1
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] O   </FolderHierarchy:Status>
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] O   <FolderHierarchy:SyncKey>
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] O   {8e6add07-3b09-4e2b-8809-7ffd9c80955c}2
    13/02/2020 15:16:11 [ 4161] [WBXML] [zpushy@doen] O   </FolderHierarchy:SyncKey>
    
    


  • I’ll look into that!



  • nope, not better, even though I made it return the exact same key (reverse chronological order, read from bottom->up):

    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O  </FolderHierarchy:FolderSync>
    18/02/2020 12:50:56 [ 3634] [DEBUG] [zpushy@doen] ASDevice->GetHierarchyCache(): HierarchyCache is up - Cached objects: 4
    18/02/2020 12:50:56 [ 3634] [DEBUG] [zpushy@doen] DeviceManager->CheckFolderData() checking integrity of hierarchy cache with synchronized folders
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   </FolderHierarchy:Changes>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O    </FolderHierarchy:Count>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O    0
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O    <FolderHierarchy:Count>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   <FolderHierarchy:Changes>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   </FolderHierarchy:SyncKey>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   {9328e8b8-b359-45b5-a510-1b770ce39035}1
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   <FolderHierarchy:SyncKey>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   </FolderHierarchy:Status>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   1
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O   <FolderHierarchy:Status>
    18/02/2020 12:50:56 [ 3634] [WBXML] [zpushy@doen] O  <FolderHierarchy:FolderSync>
    18/02/2020 12:50:56 [ 3634] [ INFO] [zpushy@doen] ExportChangesDiff->InitializeExporter(): Found '0' changes for 'hierarchy'
    
    ....
    
    18/02/2020 12:50:39 [ 3634] [WBXML] [zpushy@doen] I   </FolderHierarchy:SyncKey>
    18/02/2020 12:50:39 [ 3634] [WBXML] [zpushy@doen] I    {9328e8b8-b359-45b5-a510-1b770ce39035}1
    18/02/2020 12:50:39 [ 3634] [WBXML] [zpushy@doen] I   <FolderHierarchy:SyncKey>
    18/02/2020 12:50:39 [ 3634] [WBXML] [zpushy@doen] I  <FolderHierarchy:FolderSync>
    


  • @bdoeen said in Unable to keep appointments after foldersync:

    I am also suspecting this other post is about more or less the same problem: https://forum.kopano.io/topic/2982/unable-to-sync-contacts

    It is possible. Adding Fake Outbox and Sent Items folders appear for now to have solved the issue in the other thread. Do you have all the basic Email folders configured for the account? Inbox, Outbox, Sent, Deleted. Some clients require that they all exist even if they are not used.



  • Just added a bunch of folders and it works :D

    I’m just checking everything around it and will post a longer reply how I solved it! (I want to nail down which folders and how exactly for the next one stubmling over this)



  • Here‘s my solution and it works stable as far as my testing went:

    Problem
    This is for syncing appointments (calendar) and addresses (contacts) with Android. The first time I was syncing everything looked good, appointments showed up, addresses showed up, I even could open them and read the details. Seconds to minutes later, everything vanished again. Anything I edited was lost. It also disappeared, when I hit sync again. Thunderbird and iOS never had this problem (knock on wood).

    Solution
    In my self-written backend there‘s a function getFolderList() that returns all the folders for syncing. At first it only returned root, tasks, appointments and adresses – there was no Emailfolder, since I do not want to sync that.

    After adding

    • email-inbox (SYNC_FOLDER_TYPE_INBOX),
    • email-deleted (SYNC_FOLDER_TYPE_WASTEBASKET),
    • email-outbox (SYNC_FOLDER_TYPE_OUTBOX)
    • and email-sent (SYNC_FOLDER_TYPE_SENTMAIL)

    it worked. I now reduced that through some more testing to only the email-inbox and it‘s still syncing my appointments and addresses without a problem and without them disappearing again. EDIT: No, some more testing and it’s not stable with only the inbox. Just keep all the folders.

    My theory is that Android somehow relies on having at least the inbox folder and assumes there‘s an error if it is missing. With this idea, my logs read differently:

    Step 1: foldersync with key, gets “0 changes” - Android assumes it is out of sync, since it is missing the email-folder
    Step 2: foldersync without a key, gets the whole list and a new sync key
    Step 3: sync without key, gets a new key for appointments
    Step 4: sync with key, gets the whole data of appointments
    Step 5: foldersync with the new sync key, gets “0 changes” - Android assumes it is out of sync, since it is missing the email-folder and breaks off

    So the problem is not the last sync, which I assumed to be a rogue sync that had a problem, but already the first, since it was missing the emailfolder(s).

    Additional Info
    In my first post is a pastebin, where you can see the “before” in lines 117-198. Here’s part of the log from a successful foldersync (reverse chronological order, start reading at the bottom):

    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O  </FolderHierarchy:FolderSync>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   </FolderHierarchy:Changes>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    </FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     2
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     onOffice Email-Inbox
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooroot
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooemailinbox
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    <FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    </FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     9
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     onOffice Adressen
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooroot
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooaddress
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    <FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    </FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     8
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     onOffice Termine
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooroot
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooappointments
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    <FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    </FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     7
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     onOffice Aufgaben
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooroot
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ootasks
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    <FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    </FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     1
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:Type>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     onOffice root
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:DisplayName>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     0
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ParentId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     </FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     ooroot
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O     <FolderHierarchy:ServerEntryId>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    <FolderHierarchy:Add>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    </FolderHierarchy:Count>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    5
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O    <FolderHierarchy:Count>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   <FolderHierarchy:Changes>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   </FolderHierarchy:SyncKey>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   {81ba4da1-5c9a-48b5-b1ca-edfc8cab050d}1
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   <FolderHierarchy:SyncKey>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   </FolderHierarchy:Status>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   1
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O   <FolderHierarchy:Status>
    28/02/2020 11:01:56 [ 3704] [WBXML] [zpushy@doen] [androidc1323317619] O  <FolderHierarchy:FolderSync>
    28/02/2020 11:01:56 [ 3704] [DEBUG] [zpushy@doen] [androidc1323317619] HierarchyCache: AddFolder() serverid: ooemailinbox displayname: onOffice Email-Inbox
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::StatFolder: id=ooemailinbox
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::GetFolder: id=ooemailinbox
    28/02/2020 11:01:56 [ 3704] [DEBUG] [zpushy@doen] [androidc1323317619] HierarchyCache: AddFolder() serverid: ooaddress displayname: onOffice Adressen
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::StatFolder: id=ooaddress
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::GetFolder: id=ooaddress
    28/02/2020 11:01:56 [ 3704] [DEBUG] [zpushy@doen] [androidc1323317619] HierarchyCache: AddFolder() serverid: ooappointments displayname: onOffice Termine
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::StatFolder: id=ooappointments
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::GetFolder: id=ooappointments
    28/02/2020 11:01:56 [ 3704] [DEBUG] [zpushy@doen] [androidc1323317619] HierarchyCache: AddFolder() serverid: ootasks displayname: onOffice Aufgaben
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::StatFolder: id=ootasks
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::GetFolder: id=ootasks
    28/02/2020 11:01:56 [ 3704] [DEBUG] [zpushy@doen] [androidc1323317619] HierarchyCache: AddFolder() serverid: ooroot displayname: onOffice root
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::StatFolder: id=ooroot
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::GetFolder: id=ooroot
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] ExportChangesDiff->InitializeExporter(): Found '5' changes for 'hierarchy'
    28/02/2020 11:01:56 [ 3704] [ INFO] [zpushy@doen] [androidc1323317619] BackendOnOfficeSync::GetFolderList
    

    Big thank you for everyone who pitched in, here and in the other thread! :)

    Bye,
    BD


Log in to reply