Navigation

    Kopano
    • Register
    • Login
    • Search
    • Categories
    • Get Official Kopano Support
    • Recent
    Statement regarding the closure of the Kopano community forum and the end of the community edition

    Upgrade core 11.0.1.2 to 11.0.2.41 - High search cpu usage

    Kopano Groupware Core
    14
    36
    3531
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • naspelupo
      naspelupo @zara-kopa last edited by naspelupo

      @zara-kopa my kopano db also sits on a mariadb server, which is without the kopano-search barely active. Is the problem a mariadb thingy?
      When I stop kopano-search - which sometimes causes messages in syslog indicating it does not stop in time, the following is in the search logfile

      2021-08-10T18:58:28.492: [INFO ] search - stopping search
      2021-08-10T18:58:28.495: [ERROR ] index0 - Traceback (most recent call last):
      File “/usr/lib/python3/dist-packages/MAPICore.py”, line 662, in <lambda>
      setattr = lambda self, name, value: _swig_setattr(self, IMAPIFolder, name, value)
      File “/usr/lib/python3/dist-packages/kopano/service.py”, line 206, in <lambda>
      signal.signal(signal.SIGTERM, lambda *args: sys.exit(0))
      SystemExit: 0

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
      File “/usr/lib/python3/dist-packages/kopano/log.py”, line 140, in log_exc
      try: yield
      File “/usr/lib/python3/dist-packages/kopano_search/init.py”, line 202, in main
      if not self.indexed_folder(config, store, folder):
      File “/usr/lib/python3/dist-packages/kopano_search/init.py”, line 173, in indexed_folder
      path = folder.path
      File “/usr/lib/python3/dist-packages/kopano/folder.py”, line 250, in path
      subtree_entryid = self.store.subtree.entryid
      File “/usr/lib/python3/dist-packages/kopano/store.py”, line 242, in subtree
      return _folder.Folder(self, _benc(ipmsubtreeid))
      File “/usr/lib/python3/dist-packages/kopano/folder.py”, line 145, in init
      self.mapiobj
      File “/usr/lib/python3/dist-packages/kopano/folder.py”, line 157, in mapiobj
      self._mapiobj = self.store.mapiobj.OpenEntry(self._entryid,
      File “/usr/lib/python3/dist-packages/MAPICore.py”, line 602, in OpenEntry
      return _MAPICore.IMsgStore_OpenEntry(self, cbEntryID, lpInterface, ulFlags)
      SystemError: <built-in function IMsgStore_OpenEntry> returned a result with an error set

      KR paul

      1 Reply Last reply Reply Quote 0
      • A Former User
        A Former User last edited by

        The next weekend I’ll try the >= 11.0.2.43 on my test / DR system.

        1 Reply Last reply Reply Quote 0
        • A Former User
          A Former User last edited by

          Same problem with 11.0.2.43. Even with an empty db and empty attachments / search-folder kopano-server ~ 53%, kopano-search ~ 27% and mysqld ~ 22% CPU

          1 Reply Last reply Reply Quote 0
          • zara-kopa
            zara-kopa last edited by

            I have a new fact for my Debian 10 enviroment:

            MySQL (MariaDB 10) run in a VM on Proxmox Server. Via crontab-script there ist a daily backup-dump of the hole VM in the early morning. For this time (also via crontab) i stop the whole Kopano-Server thats resists on an other barematel server Debian 10 server. 5 minutes bevor backup-shutdown of Maria Kopano is stopped and after 30 minutes Kopano restarts again. Kopano-Search is allways on.
            After nearly each restart there is the high CPU usage. Stopping K-Search helps, but after restart of Search the high CPU usage is there again. On the last weekend i updated PROXMOX (not the MARIA VM) to Debian 11. In this time Kopano was still running but Maria was down. Several Proxmox (including the VM) restarts later everything works fine, also for the running Kopano / Maria construct. Even with high CPU. — Then i have to do a last reboot of the MariaDB. It shows a new aspect. After the reboot of Maria the CPU usage low as in earlyer installations. Search was runnig! Since then i simple do a reboot of Maria if the CPU usage is high. Thats works nearly ever.

            Try it on your maschine. For me it looks that there is problem in the connection between K-Search and Maria.

            1 Reply Last reply Reply Quote 0
            • A Former User
              A Former User last edited by

              Same problem after os-reboot. Only workaround is a fallback to the old version.

              1 Reply Last reply Reply Quote 0
              • zara-kopa
                zara-kopa last edited by

                My solution only works if the mariadb is not on the same hardware as the kopano server. Absolut reliable in my configuration. Kopano runs with search on. CPU usage on the kopano server by nearly 40%. — Shutdown the MariaDB server with the kopano-db for astimate 5 minutes. Restart the MariaDB. 5 minutes alter this CPU usage on the kopano-server goes down to astimate 12%. You can’t test it on a single (all-in) server.

                naspelupo 2 Replies Last reply Reply Quote 0
                • naspelupo
                  naspelupo @zara-kopa last edited by naspelupo

                  @zara-kopa I can confirm that a reboot of the mariadb server after starting kopano-search results in a normal cpu usage of both server and search again. On my installation I have a mail server and a maria-db server.
                  In the logfile numerous MAP_DISK_ERRORS as below and then mail and db server return to normal CPU

                  2021-08-25T06:58:16.291: [ERROR ] index0 - Traceback (most recent call last):
                  File “/usr/lib/python3/dist-packages/kopano/log.py”, line 140, in log_exc
                  try: yield
                  File “/usr/lib/python3/dist-packages/kopano_search/init.py”, line 202, in main
                  if not self.indexed_folder(config, store, folder):
                  File “/usr/lib/python3/dist-packages/kopano_search/init.py”, line 173, in indexed_folder
                  path = folder.path
                  File “/usr/lib/python3/dist-packages/kopano/folder.py”, line 250, in path
                  subtree_entryid = self.store.subtree.entryid
                  File “/usr/lib/python3/dist-packages/kopano/store.py”, line 242, in subtree
                  return _folder.Folder(self, _benc(ipmsubtreeid))
                  File “/usr/lib/python3/dist-packages/kopano/folder.py”, line 145, in init
                  self.mapiobj
                  File “/usr/lib/python3/dist-packages/kopano/folder.py”, line 157, in mapiobj
                  self._mapiobj = self.store.mapiobj.OpenEntry(self._entryid,
                  File “/usr/lib/python3/dist-packages/MAPICore.py”, line 602, in OpenEntry
                  return _MAPICore.IMsgStore_OpenEntry(self, cbEntryID, lpInterface, ulFlags)
                  MAPI.Struct.MAPIErrorDiskError: MAPI error 80040116 (MAPI_E_DISK_ERROR)

                  1 Reply Last reply Reply Quote 0
                  • naspelupo
                    naspelupo @zara-kopa last edited by

                    @zara-kopa after 6 hours without issues, kopano-search is back at excessive cpu usage. Stopping kopano-search again…

                    1 Reply Last reply Reply Quote 0
                    • zara-kopa
                      zara-kopa last edited by

                      @naspelupo

                      yes you are rigth. Without any reason k-search results in a new high cpu usage. But in my enviroment this error starts only after reveral days and not in hours. I control my network with PRTG, and create an alarm for a persistent high cpu usage over 40% for longer then 2 hours. Then i stop and restart the MariaDB server for several minutes.
                      After that cpu usage is normal for nearly a week. Everything, included k-search, runs fine.
                      Actually i have no time to get deeper in this problem.
                      “Ja die Sache nervt, bringt aber meine Hardware nicht ins Nirwana”

                      grüßing
                      Zara-Kopa

                      1 Reply Last reply Reply Quote 0
                      • icey
                        icey last edited by

                        Hi All,
                        same here. Fresh installation, just 6 users added, no big data yet as it is not yet running in “production”.
                        Core 11.0.2.43, as soon as kopano-search is started the CPU is at round about 50% continually. A service kopano-search stop takes its time and then CPU drops to 1%, everything is fine.
                        In the log of k-search:

                        Traceback (most recent call last):
                          File "/usr/lib/python3/dist-packages/kopano/log.py", line 140, in log_exc
                            try: yield
                          File "/usr/lib/python3/dist-packages/kopano_search/__init__.py", line 200, in main
                            path = folder.path
                          File "/usr/lib/python3/dist-packages/kopano/folder.py", line 250, in path
                            subtree_entryid = self.store.subtree.entryid
                          File "/usr/lib/python3/dist-packages/kopano/store.py", line 242, in subtree
                            return _folder.Folder(self, _benc(ipmsubtreeid))
                          File "/usr/lib/python3/dist-packages/kopano/folder.py", line 145, in __init__
                            self.mapiobj
                          File "/usr/lib/python3/dist-packages/kopano/folder.py", line 158, in mapiobj
                            IID_IMAPIFolder, MAPI_MODIFY)
                          File "/usr/lib/python3/dist-packages/MAPICore.py", line 602, in OpenEntry
                            return _MAPICore.IMsgStore_OpenEntry(self, cbEntryID, lpInterface, ulFlags)
                        SystemError: <built-in function IMsgStore_OpenEntry> returned a result with an error set
                        

                        Anyone an idea?
                        Thanks in advance
                        Icey

                        1 Reply Last reply Reply Quote 0
                        • icey
                          icey last edited by

                          Dear all,

                          does nobody has an idea how this could be fixed? It is weird, cause actually there is nearly nothing to be indexed by now. Nothing in attachements folder and also approxiametly 12 mails in all mailboxes.
                          Would be great if anybody could help.
                          What about the others? Probs solved?

                          Thanks in advance,
                          Icey

                          umgfoin 1 Reply Last reply Reply Quote 0
                          • JungleMarc
                            JungleMarc last edited by

                            I experience same issue.

                            I am not on mariadb, but on mysql.

                            My sql is not part of the same machine. I can confirm mysql is very stable and no performance issues. It does not appear the issue is with the database server.

                            On the application server for Kopano, yes I can reach high cpu usage for both kopano-server and kopano-search after several weeks of uptime.

                            My only workaround is a restart of kopano-server, kopano-search

                            How can we debug this further?

                            1 Reply Last reply Reply Quote 0
                            • umgfoin
                              umgfoin @icey last edited by umgfoin

                              @icey said in Upgrade core 11.0.1.2 to 11.0.2.41 - High search cpu usage:

                              does nobody has an idea how this could be fixed?

                              Hello @icey ,
                              I fixed the possible cause some time ago, but didn’t have time to create a qualified PR, yet.
                              You can resolve the problem by replacing a few lines in /usr/lib/python3.6/site-packages/kopano_search-11.0.2-py3.6.egg/kopano_search/__init__.py (the exact path on your system depends on distribution, python- and kopano-version).
                              Take a look here.

                              bg,
                              umgfoin.

                              umgfoin stan23 externa1 Joost mhope 5 Replies Last reply Reply Quote 2
                              • umgfoin
                                umgfoin @umgfoin last edited by

                                (the exact path on your system depends on distribution, python- and kopano-version).

                                In your case more likely to be found here: /usr/lib/python3/dist-packages/kopano_search/__init__.py
                                ++umgfoin.

                                1 Reply Last reply Reply Quote 1
                                • stan23
                                  stan23 @umgfoin last edited by

                                  @umgfoin that did the trick, thanks a lot!

                                  1 Reply Last reply Reply Quote 1
                                  • externa1
                                    externa1 @umgfoin last edited by

                                    @umgfoin
                                    works also for 8.7.23

                                    rg
                                    Christian

                                    1 Reply Last reply Reply Quote 1
                                    • icey
                                      icey last edited by

                                      Hi umgfoin,

                                      THANKS a lot --> That did the needed magic ;)

                                      Thanks.
                                      Icey

                                      1 Reply Last reply Reply Quote 0
                                      • Joost
                                        Joost Kopano @umgfoin last edited by Joost

                                        @naspelupo Thank you for reporting this issue, I’ve created a ticket to track it:
                                        https://jira.kopano.io/browse/KC-2005

                                        And thanks to @umgfoin’s fix, I’ve also been able to quickly create a PR:
                                        https://stash.kopano.io/projects/KC/repos/kopanocore/pull-requests/3559/overview

                                        umgfoin 1 Reply Last reply Reply Quote 0
                                        • umgfoin
                                          umgfoin @Joost last edited by

                                          @joost said in Upgrade core 11.0.1.2 to 11.0.2.41 - High search cpu usage:

                                          I’ve also been able to quickly create a PR:

                                          Yeah - and an impressively minimalistic one ;-) …
                                          Thanks!

                                          dev107 1 Reply Last reply Reply Quote 0
                                          • zara-kopa
                                            zara-kopa last edited by

                                            The fix from @umgfoin is great! Works for me either!

                                            Thanks a lot.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post