Navigation

    Kopano
    • Register
    • Login
    • Search
    • Categories
    • Get Official Kopano Support
    • Recent
    Statement regarding the closure of the Kopano community forum and the end of the community edition

    [SOLVED] "Message cannot be opened" problem with core-8.4.90.1296_0+157-Ubuntu_16.04-amd64

    Kopano Groupware Core
    10
    25
    4274
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jackson5
      jackson5 last edited by jackson5

      Hi,

      Since many years I update the community edition of zarafa/kopano core and webapp every couple of weeks, almost without any serious problems.

      Unfortunately yesterday, I found that in my mailboxes all messages received and sent within a certain time period (Nov 1st till Dec 11th) cannot be opened, nor deleted or moved. The folders containing these messages cannot be deleted as well. Older and newer messages are still ok.

      I checked this with webapp as well as with IMAP clients (mutt, Thunderbird) and plain IMAP via telnet. Mutt instantly freezes when entering a folder containing these messages.

      There are no hints in the server.log file.

      Even a kopano-backup run fails. It reports something like

      2017-12-12 15:04:10,437 - backup0 - ERROR - could not process change for entryid 8090DA040000000030000000306D5C308090DA0400000000300000000000000030000000000000006E5C306B5C30655C ([SPropValue(0x65E00102, 'u\xd3\xc7\x0e\x85\x9cI\xf2\x94@)F\x96\x1dv\x12\xaf\x1a\x02\x00\x00\x00'), SPropValue(0x30080040, 2017/12/11 19:03:36 GMT), SPropValue(0x65E20102, 'u\xd3\xc7\x0e\x85\x9cI\xf2\x94@)F\x96\x1dv\x12\xf0\x80\x0b\x00'), SPropValue(0x65E10102, 'u\xd3\xc7\x0e\x85\x9cI\xf2\x94@)F\x96\x1dv\x12\xa3*\x00\x00\x00\x00\x00\x00'), SPropValue(0x65E30102, '\x14u\xd3\xc7\x0e\x85\x9cI\xf2\x94@)F\x96\x1dv\x12\xf0\x80\x0b\x00'), SPropValue(0x0FFF0102, '\x80\x90\xda\x04\x00\x00\x00\x000\x00\x00\x000m\\0\x80\x90\xda\x04\x00\x00\x00\x000\x00\x00\x00\x00\x00\x00\x000\x00\x00\x00\x00\x00\x00\x00n\\0k\\0e\\'), SPropValue(0x67AA000B, False), SPropValue(0x0E070003, 1L), SPropValue(0x0FFA0102, 'I\xf7/\x02\x88\x06Lf\xa94\x15\x15\xe5{\x9b\x82'), SPropValue(0x67110003, 255235L), SPropValue(0x67150003, 10448L)]):
      2017-12-12 15:04:10,440 - backup0 - ERROR - Traceback (most recent call last):
        File "/usr/lib/python2.7/dist-packages/kopano/ics.py", line 104, in ImportMessageChange
          item.mapiobj = _utils.openentry_raw(mapistore, entryid.Value, 0)
        File "/usr/lib/python2.7/dist-packages/kopano/utils.py", line 74, in openentry_raw
          return _openentry_helper(mapistore, entryid, flags | MAPI_MODIFY)
        File "/usr/lib/python2.7/dist-packages/kopano/utils.py", line 68, in _openentry_helper
          return mapistore.OpenEntry(entryid, IID_IECMessageRaw, flags)
        File "/usr/lib/python2.7/dist-packages/MAPICore.py", line 965, in OpenEntry
          return _MAPICore.IMsgStore_OpenEntry(self, cbEntryID, lpInterface, ulFlags)
      MAPIErrorInvalidEntryid: MAPI error 80040107 (MAPI_E_INVALID_ENTRYID)
      

      I recognized the problem yesterday, running version 8.4.90.1296 of core on Ubuntu 16.04 . It is still the case for the current version as of today. Maybe the time period is correlated with the installation of the core version that I used prior to 8.4.90.1296 (cannot remember the exact version).

      Maybe this is a known problem? Is there a way to identify and delete these corrupted messages in the SQL database to make the folders browsable again?

      Best regards,
      jacks

      fbartels 1 Reply Last reply Reply Quote 0
      • fbartels
        fbartels Kopano @jackson5 last edited by

        Hi @jackson5 ,

        issue known and origin of the problem is fixed. https://forum.kopano.io/topic/819/kc-entryid-corruption-on-server-restart-with-mapi-error-80040107-mapi_e_invalid_entryid-in-new-or-modified-mail-items gives some pointers how such an item could be corrected, but we decided not to invest further time into a fix script for this.

        Regards Felix

        Resources:
        https://kopano.com/blog/how-to-get-kopano/
        https://documentation.kopano.io/
        https://kb.kopano.io/

        Support overview:
        https://kopano.com/support/

        1 Reply Last reply Reply Quote 0
        • jackson5
          jackson5 last edited by jackson5

          Oh no, it would be more than 100 messages.
          The proposed solution is not feasible to me, since it was suggested to search the properties table for the subject of the message, that usually not unique and leads to several hierarchy-IDs even in the unique case.

          I tried to recreate the database, but since kopano-backup fails also, this does not work.

          Moreover, I will not be able to successfully backup the mailboxes in the future.
          That would be a catastrophy to me, after all these years of using the community version and reporting bugs.

          If I move all valid messages from the affected folders, is there a way to delete a complete folder or all messages in a specified folder from the database?

          Any help would be appreciated.

          Best regards,
          jacks

          1 Reply Last reply Reply Quote 0
          • darootler
            darootler last edited by

            Same problem here, that’s a real catastrophe because this problem is causing a z-push loop…

            Please help us to get a workaround/fix.

            Regards
            Richard

            1 Reply Last reply Reply Quote 0
            • Martin
              Martin last edited by

              I have the same problem with several items.

              Best regards
              Martin

              1 Reply Last reply Reply Quote 0
              • bosim
                bosim last edited by

                Here is a script that will fix the broken entryids of messages in a folder.

                https://gist.github.com/bosim/1705c8b23d9398bed9b7177708db82ac

                It requires python-mysql-connector, read instructions in comments in the beginning of the script.

                Backup your database before running this script!

                Now you have been warned.

                Bo

                darootler 1 Reply Last reply Reply Quote 2
                • darootler
                  darootler @bosim last edited by

                  @bosim said in “Message cannot be opened” problem with core-8.4.90.1296_0+157-Ubuntu_16.04-amd64:

                  Here is a script that will fix the broken entryids of messages in a folder.

                  https://gist.github.com/bosim/1705c8b23d9398bed9b7177708db82ac

                  It requires python-mysql-connector, read instructions in comments in the beginning of the script.

                  Backup your database before running this script!

                  Now you have been warned.

                  Bo

                  Thank you very much, that works and saved my day :-)

                  Regards
                  Richard

                  bosim 1 Reply Last reply Reply Quote 0
                  • bosim
                    bosim @darootler last edited by

                    @darootler said in “Message cannot be opened” problem with core-8.4.90.1296_0+157-Ubuntu_16.04-amd64:

                    Thank you very much, that works and saved my day

                    Happy I could help!

                    Bo

                    1 Reply Last reply Reply Quote 0
                    • jackson5
                      jackson5 last edited by

                      Yeah, thanx man.

                      jacks

                      1 Reply Last reply Reply Quote 0
                      • fallingdamps
                        fallingdamps last edited by

                        Dear readers

                        Thank you all for your valuable assistance on the subject concerning entryid corruption. I still have problems with public folders and can not fix them.

                        I get the following error messages in WebApp:

                        • Could not open message.
                        • Could not find message, either it has been moved or deleted or you don’t have access to open this message.

                        My problems are not only related to mails, but also to contacts. As far as I know contacts were never mentioned before in the forum.

                        I have successfully used the script fix-entryids.py to fix problems with mails and contacts of real users. On my machine I had to install the missing package python-mysql.connector.

                        Now to my problem. I am having trouble to fix contacts in the public folders. I can not find out hierarchyid (step 4 mentioned in fix-entryids.py).

                        The python session look like this:

                        folder=kopano.Server().user(“SYSTEM”).folder(“Public Folders/Kontakte Test”)
                        Traceback (most recent call last):
                        File “<stdin>”, line 1, in <module>
                        File “/usr/lib/python2.7/dist-packages/kopano/store.py”, line 378, in folder
                        return self.subtree.folder(path, recurse=recurse, create=create)
                        AttributeError: ‘NoneType’ object has no attribute ‘folder’

                        I have run zarafa for several years and i have recently switched to kopano. This is my first posting in this forum. So please foregive me possible mistakes. And thank you all for your work and your assistance.

                        Regards
                        fallingdamps

                        bosim 1 Reply Last reply Reply Quote 0
                        • bosim
                          bosim @fallingdamps last edited by

                          @fallingdamps said in [SOLVED] “Message cannot be opened” problem with core-8.4.90.1296_0+157-Ubuntu_16.04-amd64:

                          folder=kopano.Server().user(“SYSTEM”).folder(“Public Folders/Kontakte Test”)

                          Try folder=kopano.Server().public_store.folder(“Kontakte Test”)

                          Bo

                          1 Reply Last reply Reply Quote 0
                          • fallingdamps
                            fallingdamps last edited by

                            @bosim said in [SOLVED] “Message cannot be opened” problem with core-8.4.90.1296_0+157-Ubuntu_16.04-amd64:

                            folder=kopano.Server().public_store.folder(“Kontakte Test”)

                            Thank you, this is working. Now I get hierarchyid.

                            I tried the script with the storeguid of SYSTEM and hierachyid. In WebApp the known error messages was still appearing (although the script did update the exact number of contacts in the folder).

                            And then I remembered the user who did create all (!) the contacts in this public folder. I tried it again with the storeguid of this user and now it works. Hooray, the contacts are not lost.

                            I do not know the details of the database scheme and the consequences of the SQL statements in the script. And I do not know what happens if different users had created entries in the same corrupted public folder (solution more complex?). Therefore I hope that this was a good way to fix my problem.

                            Thank you again for the quick response.

                            Regards
                            fallingdamps

                            bosim 1 Reply Last reply Reply Quote 0
                            • bosim
                              bosim @fallingdamps last edited by

                              @fallingdamps said in [SOLVED] “Message cannot be opened” problem with core-8.4.90.1296_0+157-Ubuntu_16.04-amd64:

                              And then I remembered the user who did create all (!) the contacts in this public folder. I tried it again with the storeguid of this user and now it works. Hooray, the contacts are not lost.

                              Interesting it works, because you should use the store guid for the public store, so kopano.Server().public_store.guid

                              It should be safe to run the script again with that guid and the hierarchyid of the folder in the public store.

                              Glad I could help.

                              Have a nice weekend and best regards,
                              Bo

                              1 Reply Last reply Reply Quote 0
                              • fallingdamps
                                fallingdamps last edited by fallingdamps

                                It seems that I made a mistake. I used the following statement in step 3 to produce a wrong parameter:

                                • kopano.Server().user(“SYSTEM”).store.guid

                                I have now used the following and hopefully correct statements thanks to bosim:

                                • kopano.Server().public_store.guid (step 3)
                                • kopano.Server().public_store.folder(“Kontakte Test”).hierarchyid (in step 4)

                                This solution may be obvious for developers and insiders but it was not for me.

                                It is perhaps a good idea if someone could add this to the comments in the script fix-entryids.py.

                                And now finally: have a nice weekend, too.

                                Regards
                                fallingdamps

                                bosim 1 Reply Last reply Reply Quote 0
                                • bosim
                                  bosim @fallingdamps last edited by

                                  @fallingdamps said in [SOLVED] “Message cannot be opened” problem with core-8.4.90.1296_0+157-Ubuntu_16.04-amd64:

                                  This solution may be obvious for developers and insiders but it was not for me.
                                  It is perhaps a good idea if someone could add this to the comments in the script fix-entryids.py.
                                  And now finally: have a nice weekend, too.

                                  Well you are right it could have been more simple, but I am happy you made it working. We do not expect more of these incidents to come, it was a very unfortunate situation having the database in a state where it could not be easily repaired by kopano-backup or kopano-fsck.

                                  Have a nice weekend
                                  Bo

                                  1 Reply Last reply Reply Quote 0
                                  • Sinux
                                    Sinux last edited by

                                    Thanks @bosim for your solution!!!

                                    1 Reply Last reply Reply Quote 1
                                    • Martin
                                      Martin last edited by

                                      Many thanks @bosim for your solution!
                                      All items are visible again :-)

                                      Best regards and merry xmas
                                      Martin

                                      1 Reply Last reply Reply Quote 1
                                      • tueftler17
                                        tueftler17 last edited by

                                        Hi together!

                                        After searching for hours i found this posting. Many thanks for writing this script! I had so much trouble with my wife because of missing mails, now most of them are back again.

                                        I have one question: I have to run the script on the “Sent Items” folder, but I do not know how to access it to get the hierarchy ID. I thought about searching in the hierarchy-table for the parent of the id of the inbox, looked down into the hierarchies with matching parent, but I endet in not knowing where to find in the database the matching name, so im stuck here.

                                        All I tried around this gave errors (without blanks, other quotes and so on).
                                        folder=kopano.Server().user(“xxx”).inbox.folder(“Sent Items”)
                                        folder=kopano.Server().user(“xxx”).inbox.folder(“Gesendete Objekte”)
                                        The error is always like: kopano.errors.NotFoundError: no such folder: <what I tried>

                                        When I try
                                        folder=kopano.Server().user(“xxx”).outbox
                                        and get the id
                                        the script does nothing :-(.

                                        I would really appreciate if somebody could give me a hint,

                                        kind regards,
                                        Christian

                                        darootler 1 Reply Last reply Reply Quote 0
                                        • darootler
                                          darootler @tueftler17 last edited by

                                          @tueftler17

                                          Try this:

                                          folder=kopano.Server().user("xxx").folder("Sent Items")
                                          

                                          Regards
                                          Richard

                                          1 Reply Last reply Reply Quote 0
                                          • tueftler17
                                            tueftler17 last edited by

                                            @Richard:
                                            Perfect, as usual the solution was easy but in Germany we say: “I had tomatoes on my eyes”, so I did not find it.

                                            Many thanks to you, now almost all messages are back again. I still see some errors when I do a kopano-backup but I will try to ignore them. Hopefully they won’t do a message-loop with active-sync.

                                            Again: It was very helpful, I wish you all a great 2018!

                                            kind regards,
                                            Christian

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post