Kopano-search running for couple of hours then kopano-server crashes

Hi everyone

I manager to import a huge mailbox 40gb. I needed to disable kopano-search becaus server would crash. Now all is imported and i start kopano-search however i find that all ram and swap memory is Being used.

I installed the latest nightly builds but same issue.

Does anyone have the same problem. With my small mailboxes this would not happen(that my server crashes after a few hours)

erver.log show some errors

Tue Oct 23 17:47:03 2018: [error  ] SerializeObject failed with error code 0x80000004 for object 2039934
Tue Oct 23 17:53:59 2018: [error  ] SerializeObject failed with error code 0x80000004 for object 2037209
Tue Oct 23 17:58:59 2018: [error  ] SerializeObject failed with error code 0x80000004 for object 2037027
Tue Oct 23 18:10:36 2018: [error  ] SerializeObject failed with error code 0x80000004 for object 2031524
Tue Oct 23 18:20:09 2018: [error  ] SerializeObject failed with error code 0x80000004 for object 2028513
Tue Oct 23 18:35:10 2018: [error  ] SerializeObject failed with error code 0x80000004 for object 2025117
Tue Oct 23 18:42:32 2018: [error  ] SerializeObject failed with error code 0x80000004 for object 2023191

search.log:

./customXml/item2.xml:1: namespace warning : xmlns: URI 303d96d4-3bb1-4b0b-8599-b2884526dbd3 is not absolute
 xsi:nil="true"/><IT_x0020_Solution xmlns="303d96d4-3bb1-4b0b-8599-b2884526dbd3"
                                                                               ^
./customXml/item2.xml:1: namespace warning : xmlns: URI 303d96d4-3bb1-4b0b-8599-b2884526dbd3 is not absolute
6dbd3" xsi:nil="true"/><Subcategory xmlns="303d96d4-3bb1-4b0b-8599-b2884526dbd3"
                     ..
2018-10-23 20:40:32,136 - index0 - WARNING - output on stderr:
parser error : out of memory error
parser error : out of memory error
parser error : out of memory error
parser error : out of memory error
./xl/worksheets/sheet4.xml:2: parser error : out of memory error
67" s="1" t="s"><v>65</v></c><c r="AF4767" s="1"><v>1</v></c><c r="AG4767" s="1"
                                                                               ^
parser error : out of memory error
./xl/worksheets/sheet4.xml:2: parser error : out of memory error
67" s="1" t="s"><v>65</v></c><c r="AF4767" s="1"><v>1</v></c><c r="AG4767" s="1"
                                                                               ^
unable to parse ./xl/worksheets/sheet4.xml

versions are:

kopano-server-8.6.82.110-117.1.x86_64
kopano-search-8.6.82.110-117.1.x86_64

centos version:

Linux kopano 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

fbartels

That does sound like https://forum.kopano.io/topic/1705/kopano-server-crashes/

Seems there are still some leaks left.

@fbartels

indeed seems like the same. can i help in some way to find the errors?

@fbartels

A question can we see how far indexing is somehow??

fbartels

@ckruijntjens said in Kopano-search running for couple of hours then kopano-server crashes:

A question can we see how far indexing is somehow??

No, that is not possible.

@fbartels

Hi is there Some wat then to tell when indexing is finishes? When i gestart my server Will iT pick up where iT is left or is iT starting from the begining?

fbartels

Once indexing has finished kopano-search will create the search socket. if you stop search during indexing it will continue from where it stopped.

@fbartels

hi and thank you for your reply. where should this file be created?

fbartels

By default /var/run/kopano/search.sock

fbartels

@ckruijntjens said in Kopano-search running for couple of hours then kopano-server crashes:

kopano-server-8.6.82.110-117.1.x86_64
kopano-search-8.6.82.110-117.1.x86_64

this version is already more than a week old. Can you check if you also have the same issue with the most recent build?

mark dufour

would it be possible to try and run valgrind on kopano-server, for a few minutes (note that it will become like 10 times slower)? we use valgrind internally to look for leaks, but unfortunately we can’t cover every possible situation, so it would be much appreciated if you could give this a try. usually if there’s any leak (not “unbounded allocation”), valgrind will find it.

@mark-dufour

Hi i Will try this and report the outcome. Wat is best for performance attachments inside database or outside?

fbartels

Definitely outside the database

@fbartels

Hi can i stil switch to outside database? Without losing everything?

fbartels

https://documentation.kopano.io/kopanocore_administrator_manual/upgrading.html#migrating-database-attachments-to-files