Z-Push / KOE not syncing anymore

Sebastian

Beto a WBXML log. The wiki has more details (debugging).

icey

Hi Manfred and Sebastian,

i would like to provide you the wbxml.log. i am not getting an idea out of it :(
Hopefully you can find an error where i can solve the problem.

Thanks in advance

Markus

Manfred

Hi Markus,

yes, you can post the link to the WBXML log and we will take a look at it.

Manfred

icey

Great, thanks! There you go: https://sapper-plus.de/log/gustav.rar

Manfred

Hi Markus,

I’ve looked through the log and didn’t find anything obviously wrong. I have some questions though.

There are quite some folders and messages you’re syncing. Have you tried synchronising 1 month instead of all items?

What is the size of the .ost file associated to this Outlook profile?

The both broken appointments, “Internorga 2013” and “Sapper Göppingen” have a quite unusual UID. Are they invitations from external system?

Have you tried running kopano-fsck?

Manfred

icey

Hi Manfred,
thanks a lot so far. These two appointments had as date 01.01.1753. Why? I don’t know… May you tell how you found these? How did you search? I now get it synct up to 33% and it gets stucked again… So i would like to give it a try to find it by myself.
You talked about kopano-fsck. don’t know this tool yet. when i make a --checkonly i get the following results:
Statistics:

IPF.Appointment
        Folders:        1
        Entries:        541
        Problems:       38
        Fixed:          0
        Deleted:        0
IPF.Contact
        Folders:        13
        Entries:        3387
        Problems:       293
        Fixed:          0
        Deleted:        0
IPF.Task
        Folders:        1
        Entries:        13
        Problems:       0
        Fixed:          0
        Deleted:        0

Should i try a --autofix or what would you suggest? I can’t just delete all the contacts and so on…

Thanks in advance
Markus

Manfred

Hi Markus,

I was looking for “potential broken message id” in the log and noticed the unusual uid. Now that you say the timestamp is pretty strange indeed as well.

I would suggest you run kopano-fsck for calendar only without --autofix. If I remember correctly it will show you what it would fix, so that you can decide if you want to accept it or not.
There are quite some contacts with problems, so running without --autofix might be quite tedious.
Anyway, I think kopano-fsck should get you further in sync.

Is it actually an older installation with some updates of Zarafa/Kopano or have you migrated from some other solution?

Manfred

icey

Hi Manfred,
no it is a running installation since years now. Originally it was Zarafa, then migrated to kopano, and since a few reinstallation and server movings it runs on that server since 2 years i think. without any probs so far. just i think after the last upate the probs began…
So what is the indication for an unusual id? when it ends with “2000000” so then there would be many…
kopano-fsck without an option asks if it should delete the item or not… a bit annoying to type 38 times “yes” \o/ ^^

Regards
Markus

Manfred

Hi Markus,

the usual uids start with 0400000082… (e.g. 040000008200E00074C5B7101A82E0080000000000734468419CD301000000000000000010000000F59DB9B0B89CD0B831193088D7E52010).

But it might be also other issues with the items, not uid only (like end before start or property value out of range defined in ActiveSync). It’s not easy to spot such issues.

I didn’t remember that kopano-fsck was deleting items. But if you don’t need them anymore, that’s probably the way to go.

Manfred

icey

Hi Manfred,
i come back to you because i can´t get the prob fixed. So i tried with kopano-fsck for the appointments, but it didn´t help. So in outlook it says “Folder Inbox is being synced” but it stucks… For what may i search in the gustav.log ?
Further i recognized that i got errors in server.log finding some attachements. Nobody deleted anything…
Also in webapp only at gustav it shows some curious errror messages in setting like: “can not get list of delegates” and when saving an appointement “could not save the message” but it is saved(sometimes)…?!

Another thing on my own installation and that installation is, when i create a contact on the iphone which is connected via zpush, in webapp is shown at the field fullname:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"><HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><META NAME="Generator" CONTENT="Kopano HTML builder 1.0"><TITLE></TITLE></HEAD><BODY><!-- Converted from text/plain

Sorry this is very much at one time i know \o/ Hopefully you are able to give me some hints, I will try figuring the errors out on saturday, because i got very less time at the moment…

Thanks a lot in advance!

Cheers Markus

Coffee_is_life

Good Morning @icey,

since its “only” this store, you could try to

backup the store with kopano-backup

kopano-backup -u gustav -O <output-dir>

unhook the store

kopano-admin --unhook-store gustav

now you got 2 options:
3 a) disable gustav for e-mail (for ldap, disable the user in ad), wait for kopano to sync

kopano-admin -l      (gustav not shown)

enable again, wait for kopano to get this change aswell. it should create a new store for gustav

or 3 b) create a store manually

kopano-admin --create-store gustav

and finally restore the backup

kopano-backup --restore <previous_output_folder>/gustav

if this is successful you can delete the orphaned store

kopano-admin --list-orphans (gustav should be guessed by kopano, you can see its id)
kopano-admin -remove-store <orphaned_id>

if this does not work, try to fix the folder ids by running

python fix_delegates.py --user gustav --modify

(https://stash.kopano.io/projects/KSC/repos/core-tools/browse/fix-delegates)

hope you can fix it

coffee_is_life

icey

Hey coffee_is_life,

thanks for your input. Tomorrow i will give it a new try.
Do you have any ideas belonging to the contacts prob with the full name field?
And if i have understood evrything correctly, your suggestion refers to the delegate prob, right?

Cheers
Markus

icey

Hey guys,

i really confused now… Even the backup didn´t work, cause of missing attachements… How can i quickly check if all objects got their attachement? How is it possible that there are some missing?

backup - WARNING - no data found for attachment of item with entryid 000000007CFD44A4137E443481D895849268F2E90100000005000000F332521E2B824D20AEDAD05CEFF1D25B00000000

The delegate solution either did not work :(

The new ids are saved for user gustav
Traceback (most recent call last):
  File "fix-delegates.py", line 71, in <module>
    main()
  File "fix-delegates.py", line 68, in main
    user.store.root.prop(0x36E41102).set_value(newid)
AttributeError: 'Property' object has no attribute 'set_value'

Is it possible to find out which object is which mail?

The prob with the delegates and the calendar is only at the store gustav…

Thanks in advance
Markus

Manfred

Hi Markus,

@icey said in Z-Push / KOE not syncing anymore:

Hi Manfred,
i come back to you because i can´t get the prob fixed. So i tried with kopano-fsck for the appointments, but it didn´t help. So in outlook it says “Folder Inbox is being synced” but it stucks… For what may i search in the gustav.log ?
Further i recognized that i got errors in server.log finding some attachements. Nobody deleted anything…

Well, for errors, items in loop, broken items. There’s no certain check list as there might be different issues.
That might be why outlook is stuck. It tries to get the attachment, but because it’s not available, it won’t continue to sync.
Are attachments saved on the file system or in the database? I’m not sure if there’s a python script to check for missing attachments. That would be the question for support or Kopano core section.

Also in webapp only at gustav it shows some curious errror messages in setting like: “can not get list of delegates” and when saving an appointement “could not save the message” but it is saved(sometimes)…?!

That would be a question for webapp team.

Another thing on my own installation and that installation is, when i create a contact on the iphone which is connected via zpush, in webapp is shown at the field fullname:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"><HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><META NAME="Generator" CONTENT="Kopano HTML builder 1.0"><TITLE></TITLE></HEAD><BODY><!-- Converted from text/plain

Do you have a WBXML log of such a contact being synced to the server?

It looks like the store of this user is pretty broken. Did you backup and restore it at some point?

Manfred

icey

Hi Manfred,

after deleting all corrupted contacts/appointments and E-Mails, adding missing attachements to the directories the backup went through. Then i renamed the store, deleted the whole store (about50GB…), created a new one, attached the backup to this store. Then all issues where solved. The missing data i have to recover by hand out of the snapshot… It is very much work, but i don´t know any other way.
How can i identify a mail by the object id?

The prob with the contacts is solved, have a look here: https://forum.kopano.io/topic/1164/cotacts-full-name-displayed-as-thml-code/9

Thanks so far a lot!

Cheers Markus

Manfred

Hi Markus,

@icey said in Z-Push / KOE not syncing anymore:

How can i identify a mail by the object id?

That is not possible within MAPI and ActiveSync. The id is generic and doesn’t contain information about the object type.
You either have to know the folder which contains the object and if it’s an email folder, then the object is an email (theoretically it’s possible to put calendar or contact items into an email folder, but that would be rather unusual). Or you open the object and check its class.

Manfred