Navigation

    Kopano
    • Register
    • Login
    • Search
    • Categories
    • Get Official Kopano Support
    • Recent
    Statement regarding the closure of the Kopano community forum and the end of the community edition

    "To-Do List" stops displaying new items, searching for items still works

    Kopano WebApp
    3
    10
    1071
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • con356
      con356 last edited by

      Hi forum,
      i am not sure how to explain this one correct…

      If i flag emails or enter new entries in “Tasks” no new entry is shown in the “To-Do List”. But if i search for those newly made or flagged items, then they are shown as a single search-result.

      The “To-Do List” shows older entries correct. The List just stops working…

      As far as i remember: A few days ago i changed language settings in /etc/default/kopano

      #KOPANO_LOCALE="C"
      KOPANO_LOCALE="de_DE.UTF-8"
      
      #KOPANO_USERSCRIPT_LOCALE="C"
      KOPANO_USERSCRIPT_LOCALE="de_DE.UTF-8"
      

      Changing this back did not help. May be this has nothing to do with the problem.

      This problem is effective i every user account.

      Could somebody “push” me in the right direction?

      Server: Debian_9 64bit
      core-8.6.80.3_0+79-Debian_9.0-amd64
      webapp-3.4.7.1275+733-Debian_9.0-all

      Regards
      Enno

      1 Reply Last reply Reply Quote 0
      • Coffee_is_life
        Coffee_is_life last edited by Coffee_is_life

        Hello @con356,

        this issue seems related to system folders-bug.
        try this script on a testuser and systemfolder ‘task’
        https://stash.kopano.io/projects/KSC/repos/core-tools/browse/recreate-systemfolders

        //EDIT: its a bit strange that all users are effected, thats why try with a testuser

        btw if you want to rename your stores to german locale, you need to run
        https://stash.kopano.io/projects/KSC/repos/core-tools/browse/localize-folders
        in order to rename existing stores

        coffee_is_life

        1 Reply Last reply Reply Quote 0
        • con356
          con356 last edited by

          Hi coffee_is_life,

          thank’s for help!
          I made a full clone of the whole debian-system, so any test is without risk. :-)
          “recreate-systemfolders” dosen’t seem to made a change to nothing … !?

          python recreate-systemfolders.py --user test --create Tasks --systemfolder task
          User('test')
          create tmp folder Tasks
          Promote new folder to be a system folder
          

          But there is no difference in Systemfolder ‘Tasks’ nor in Folder ‘To-Do List’

          I give it a try with an other name:

          recreate-systemfolders.py --user test --create Tasks_new --systemfolder task
          User('test')
          create tmp folder Tasks_new
          Promote new folder to be a system folder
          

          The new folder ‘Tasks_new’ was created as an normal empty(!) folder. The old folder Tasks is still there with the same tasks inside as before. No difference ether in ‘To-Do List’.

          By the way: if I flag an eMail in webapp --> in a z-push device the new task rise up as expected.

          And thank you for the hint to localize-folders! It works as it should.

          Any idea?

          con356

          1 Reply Last reply Reply Quote 0
          • Coffee_is_life
            Coffee_is_life last edited by

            @con356,

            i got a similar error with some random mails in random users.
            they are synced to z-push devices, vanishes without a trace at some point.
            if i mark them in webapp or even just click them, they are synced again to the device

            i had a ticket for that, but its still unresolved due to the randomness.
            Kopano-support said, that this might be related to the exporter, triggered by kopano-server, so that z-push gets the info which items it should sync… but its not confirmed…

            since im the only one answered here i dont think it can be solved that easily… if you have a subscription i advice to create a ticket

            but we can try here aswell :)
            try to backup the user, unhook the store, create empty store for this one and restore the items

            kopano-backup -u test
            kopano-admin --unhook-store test
            kopano-admin --create-store test 
            kopano-backup --restore -u test
            

            this is just guessing but might help :)

            coffee_is_life

            1 Reply Last reply Reply Quote 0
            • con356
              con356 last edited by

              Hi coffee_is_life,

              yes you’re right. This is not easy without the subscription. But your answer makes me feel not so lost and alone. ;-)
              I’ve got the same idea to restore a backup, while i walk around in this forum reading posts. I’ve created an new user and restore the backup of an existing one:

              kopano-backup --restore user_old -u user_new --recursive
              

              This is a solution. The new marked (flagged) eMails were now displayed immediately in ‘To-Do List’. But then i have a lot of work for all users:

              • 1.: Reset every profile on mobile devices
              • 2.: Reset every permission on calendars etc. on every user
              • … Reset public folders
              • etc.

              Hmm, wait…

              --unhook-store
              --create-store
              

              OK, this makes sense, not to have all the work. Great! :-)

              I will try few things first, with your named scipts in second post. Maybe switching languages in combination with z-push / public folders will cause the problem.

              Other solutions will be listed here.

              con356

              1 Reply Last reply Reply Quote 0
              • Coffee_is_life
                Coffee_is_life last edited by Coffee_is_life

                hello @con356,

                in case you got tons of users a simple scripting solution would do it:

                for e in $(kopano-admin -l |awk -F" " '{print $1}' | sed -e 1,4d); do 
                  echo "kopano-backup -u $e" 
                  echo "kopano-admin --unhook-store $e" 
                  echo "kopano-admin --create-store $e"
                  echo "kopano-backup --restore -u $e"
                done
                

                i’ve put everything in echos, so you can test the output of this little script.
                its untested, so i dont know if the command will wait for the backup etc…

                you can test this by changing

                for e in $(kopano-admin -l |awk -F" " '{print $1}' | sed -e 1,4d)
                

                to:

                for e in $(kopano-admin -l |awk -F" " '{print $1}' | sed -e 1,4d |grep test)  #  for a testing user named test
                

                if this solves your problems you can safely remove all stores listed under

                kopano-admin --list-orphans
                

                by simpyl using:

                for e in $(kopano-admin --list-orphans |grep private |awk -F" " '{print $1}');  do 
                  echo "kopano-admin -remove-store $e"
                done
                

                same here with the echos
                :thumbsup:

                coffee_is_life

                1 Reply Last reply Reply Quote 0
                • con356
                  con356 last edited by

                  hi @coffee_is_life,

                  i checked it again: yes, every user is effected. But there are only less then 10 of them. Puh!

                  Will try your scipts this weekend.
                  :sunglasses: :thumbsup:

                  con356

                  1 Reply Last reply Reply Quote 0
                  • con356
                    con356 last edited by

                    Finaly everything is all right now. But i have to rebuild every user with kopano-backup. Step by step i did the following:

                    1.: Be sure to have a valid backup (mysqldump / rsync attachments)

                    2.: Create a script for backup webapp-settings

                    /usr/local/bin/webapp_settings.py
                    
                    #!/usr/bin/env python
                    #encoding: utf-8
                    # https://stash.kopano.io/projects/KSC/repos/webapp-tools/browse/webapp_settings.py
                    
                    from MAPI import *
                    from MAPI.Util import *
                    import sys
                    try:
                        import kopano
                    except ImportError:
                        import zarafa as kopano
                    try:
                        import json
                    except ImportError:
                        import simplejson as json
                    
                    def opt_args():
                        parser = kopano.parser('skpcfm')
                        parser.add_option("--user", dest="user", action="store", help="Run script for user")
                        parser.add_option("--backup", dest="backup", action="store_true", help="Backup webapp setting ")
                        parser.add_option("--restore", dest="restore", action="store_true", help="Restore webapp settings")
                        parser.add_option("--remove", dest="remove", action="store_true", help="Remove webapp settings")
                    
                        return parser.parse_args()
                    
                    
                    def main():
                        options, args = opt_args()
                    
                        if not options.user or (not options.backup and not options.restore and not options.remove):
                            print 'Please use:\n %s --user <username> (--backup or --restore)  ' % (sys.argv[0])
                            sys.exit()
                    
                        user = kopano.Server(options).user(options.user)
                        if options.backup:
                            webapp = json.loads(user.store.prop(PR_EC_WEBACCESS_SETTINGS_JSON).value)
                            f = open('%s.json' % user.name,'w')
                    
                            f.write(json.dumps(webapp, sort_keys=True,
                                         indent=4, separators=(',', ': ')))
                            f.close()
                        if options.restore:
                            with open('%s.json' % user.name) as data_file:
                                data = json.load(data_file)
                    
                            # print data
                            user.store.create_prop(PR_EC_WEBACCESS_SETTINGS_JSON, json.dumps(data))
                            print 'restore done'
                        if options.remove:
                            user.store.delete(user.store.prop('PR_EC_WEBACCESS_SETTINGS_JSON'))
                    
                    if __name__ == "__main__":
                        main()
                    

                    Then i use the script coffee_is_life posted:

                    /usr/local/bin/kopano-recreate-user.sh
                    
                    #! /bin/bash
                    BACKUPPATH=/backup/kopano/full/
                    for e in $(kopano-admin -l |awk -F" " '{print $1}' | sed -e 1,4d); do
                      kopano-backup -u $e --output-dir $BACKUPPATH
                      cd $BACKUPPATH$e
                      python /usr/local/bin/webapp_settings.py --backup --user=$e
                      kopano-admin --unhook-store $e
                      kopano-admin --create-store $e
                      kopano-backup --restore $BACKUPPATH$e -u $e --recursive 
                      python /usr/local/bin/webapp_settings.py --restore --user=$e
                    done
                    

                    Now almost everything is in place again. Manualy the following has to be done:

                    3: Reset colors of calendars (own / imported) in webapp of every user

                    4.: If there were public-folders like calender/tasks/notes shared over z-push with the script soap-folders.php

                    <?php
                    
                    $server = "http://localhost";
                    $user = "user1";
                    $pass = "user1";
                    
                    $devid = "n2lmve8dm51a94esgor1gav65l";
                    
                    define("SYNC_FOLDER_TYPE_USER_MAIL", 12);
                    define("SYNC_FOLDER_TYPE_USER_APPOINTMENT", 13);
                    define("SYNC_FOLDER_TYPE_USER_CONTACT", 14);
                    define("SYNC_FOLDER_TYPE_USER_TASK", 15);
                    define("SYNC_FOLDER_TYPE_USER_JOURNAL", 16);
                    define("SYNC_FOLDER_TYPE_USER_NOTE", 17);
                    define("FLD_FLAGS_REPLYASUSER", 1);
                    
                    $client = new SoapClient(null, array(
                                    'location' => "$server/Microsoft-Server-ActiveSync?Cmd=WebserviceDevice&DeviceId=webservice&DeviceType=webservice&User=$user",
                                    'uri' => "http://localhost/soap",
                                    'trace' => 1,
                                    'login' => $user,
                                    'password' => $pass
                                    ));
                    
                    try {
                        $add_user = "user2";
                        $add_id = "5a37a3f4faa340e49f5c0dc09cf6cb043a0000000000";
                        $add_name = "User2 - Inbox";
                        $add_type = SYNC_FOLDER_TYPE_USER_MAIL;
                        $add_flags = FLD_FLAGS_REPLYASUSER;
                    	
                    //    $r = $client->AdditionalFolderAdd($devid, $add_user, $add_id, $add_name, $add_type, $add_flags);
                    //    echo "Add:\n". print_r($r, true) ."\n";
                    
                    //    $r = $client->AdditionalFolderEdit($devid, $add_id, "! Different name", $add_flags);
                    //    echo "Edit:\n". print_r($r, true) ."\n";
                    
                    
                    //    $r = $client->AdditionalFolderRemove($devid, $add_id);
                    //    echo "Remove: ". print_r($r, true) ."\n";
                    
                        $r = $client->AdditionalFolderList($devid);
                        echo "Current folders:\n". print_r($r, true);
                    }
                    catch (SoapFault $sf) {
                        echo "soap fault: ". $sf->getMessage() . "\n";
                        exit(1);
                    }
                    

                    4a.: List the old IDs

                    z-push-admin -a list -u <user>
                    

                    4b.: Remove the old IDs with soap-folders.php
                    4c: List the new IDs

                    php /usr/share/z-push/backend/kopano/listfolders.php -l <user>
                    

                    4d.: Add the new IDs with soap-folders.php
                    4e.: Reset settings in mobile devices for shared folders

                    5.: Remove the old stores

                    kopano-admin --list-orphans  # List Store GUID
                    kopano-admin --remove-store <Store GUID>
                    

                    /@coffee_is_life: Thanks for help!!

                    1 Reply Last reply Reply Quote 1
                    • fbartels
                      fbartels Kopano last edited by fbartels

                      For everyone else reading this: this is/was a known bug in Kopano 8.5.x (and therefore the master builds as well). Yesterday we merged a fix which still needs to pass final checks, afterwards we are going to release an updated 8.5 release.

                      @con356 said in "To-Do List" stops displaying new items, searching for items still works:

                      kopano-backup --restore $BACKUPPATH$e -u $e --recursive
                      python /usr/local/bin/webapp_settings.py --restore --user=$e

                      btw. kopano-backup also backs up and restores webapp settings, so this is not necessary

                      Regards Felix

                      Resources:
                      https://kopano.com/blog/how-to-get-kopano/
                      https://documentation.kopano.io/
                      https://kb.kopano.io/

                      Support overview:
                      https://kopano.com/support/

                      1 Reply Last reply Reply Quote 0
                      • Coffee_is_life
                        Coffee_is_life last edited by

                        @con356

                        glad i could help and everything is fine now

                        coffee_is_life

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post