libkcclient.so causing Apache Segmentation fault
-
Date Seen
16.10.2017Versions
Ubuntu 16.04.3 LTS
Kopano 8.4.90.492
Apache 2.4.18
MySQL 5.7.19Bug Description
Since upgrading to version 8.4.90.492 i am constantly receiving the following error message:AH00051: child pid XXXX exit signal Segmentation fault (11), possible coredump in /var/cache/apache2/core
Here is a Apache coredump:
[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/sbin/apache2 -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f78c6ae4e88 in ?? () from /usr/lib/kopano/libkcclient.so [Current thread is 1 (Thread 0x7f78ee178780 (LWP 2017))] (gdb) bt #0 0x00007f78c6ae4e88 in ?? () from /usr/lib/kopano/libkcclient.so #1 0x00007f78c6ae3d9f in ?? () from /usr/lib/kopano/libkcclient.so #2 0x00007f78c6ae3f49 in ?? () from /usr/lib/kopano/libkcclient.so #3 0x00007f78d91f978b in KC::ECUnknown::Suicide() () from /usr/lib/libkcmapi.so.0 #4 0x00007f78d91f9b70 in KC::ECUnknown::Release() () from /usr/lib/libkcmapi.so.0 #5 0x00007f78c6a95fc4 in ?? () from /usr/lib/kopano/libkcclient.so #6 0x00007f78c6a96039 in ?? () from /usr/lib/kopano/libkcclient.so #7 0x00007f78d91f978b in KC::ECUnknown::Suicide() () from /usr/lib/libkcmapi.so.0 #8 0x00007f78d91f9b70 in KC::ECUnknown::Release() () from /usr/lib/libkcmapi.so.0 #9 0x00007f78d944b179 in ?? () from /usr/lib/libmapi.so.1 #10 0x00007f78d94446ea in ?? () from /usr/lib/libmapi.so.1 #11 0x00007f78d9444749 in ?? () from /usr/lib/libmapi.so.1 #12 0x00007f78d943785a in ?? () from /usr/lib/libmapi.so.1 #13 0x00007f78e8e7b341 in ?? () from /usr/lib/apache2/modules/libphp7.0.so #14 0x00007f78e8e7b3ca in list_entry_destructor () from /usr/lib/apache2/modules/libphp7.0.so #15 0x00007f78e8e772a6 in zend_hash_index_del () from /usr/lib/apache2/modules/libphp7.0.so #16 0x00007f78e8e9de7b in zend_object_std_dtor () ---Type <return> to continue, or q <return> to quit--- from /usr/lib/apache2/modules/libphp7.0.so #17 0x00007f78e8ea2f8a in zend_objects_store_del () from /usr/lib/apache2/modules/libphp7.0.so #18 0x00007f78e8e59a33 in zend_cleanup_user_class_data () from /usr/lib/apache2/modules/libphp7.0.so #19 0x00007f78e8e55d20 in shutdown_executor () from /usr/lib/apache2/modules/libphp7.0.so #20 0x00007f78e8e660fb in zend_deactivate () from /usr/lib/apache2/modules/libphp7.0.so #21 0x00007f78e8e04132 in php_request_shutdown () from /usr/lib/apache2/modules/libphp7.0.so #22 0x00007f78e8efcb47 in ?? () from /usr/lib/apache2/modules/libphp7.0.so #23 0x0000561a6aff6c80 in ap_run_handler (r=r@entry=0x7f78edf970a0) at config.c:169 #24 0x0000561a6aff7206 in ap_invoke_handler (r=r@entry=0x7f78edf970a0) at config.c:433 #25 0x0000561a6b00e6a2 in ap_process_async_request (r=r@entry=0x7f78edf970a0) at http_request.c:410 #26 0x0000561a6b00e850 in ap_process_request (r=r@entry=0x7f78edf970a0) at http_request.c:445 #27 0x0000561a6b00a91e in ap_process_http_sync_connection (c=0x7f78ee089290) at http_core.c:210 #28 ap_process_http_connection (c=0x7f78ee089290) at http_core.c:251 ---Type <return> to continue, or q <return> to quit--- #29 0x0000561a6b000bf0 in ap_run_process_connection (c=c@entry=0x7f78ee089290) at connection.c:41 #30 0x0000561a6b001000 in ap_process_connection (c=c@entry=0x7f78ee089290, csd=<optimized out>) at connection.c:213 #31 0x00007f78e94517e9 in child_main (child_num_arg=child_num_arg@entry=5, child_bucket=child_bucket@entry=0) at prefork.c:723 #32 0x00007f78e9451a74 in make_child (s=0x7f78ee155470, slot=5, bucket=0) at prefork.c:824 #33 0x00007f78e94528c5 in perform_idle_server_maintenance (p=<optimized out>) at prefork.c:932 #34 prefork_run (_pconf=<optimized out>, plog=<optimized out>, s=<optimized out>) at prefork.c:1128 #35 0x0000561a6afd946e in ap_run_mpm (pconf=0x7f78ee183028, plog=0x7f78ee150028, s=0x7f78ee155470) at mpm_common.c:94 #36 0x0000561a6afd2400 in main (argc=3, argv=0x7ffeb39a6d38) at main.c:777
Severity
MajorSteps to Reproduce
I cannot reproduce the error, seems it has something todo with z-push.Actual Behavior
Z-Push synchronization periodically stopsExpected Behavior
Z-Push synchronization is working, no erros are reportedTroubleshooting/Testing Steps Attempted
Apache coredump attached, a similar ticket was reported here --> https://jira.kopano.io/browse/KC-259Workaround
No working workaround found yet. -
Hello @darootler ,
can zou provide a wbxml of around the time the segfault was triggered in apache?
-
Hi @fbartels
I have to cut off the log because i cannot post all the lines due to forum limitations.
Here is the wbxml log:
Oct 19 13:26:23 server kopano-server[1066]: Error while connecting to search on "file:///var/run/kopano/search.sock" Oct 19 13:26:23 server kopano-server[1066]: Error while connecting to search on "file:///var/run/kopano/search.sock" Oct 19 13:26:24 server apache: [webmail.external.domain] 192.168.1.4 - USER@internal.domain [19/Oct/2017:13:26:23 +0200] "GET / HTTP/1.1" 200 7871 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" Oct 19 13:26:24 server apache: [webmail.external.domain] 192.168.1.4 - USER@internal.domain [19/Oct/2017:13:26:24 +0200] "POST /kopano.php?service=fingerprint HTTP/1.1" 200 - "https://webmail.external.domain/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" Oct 19 13:26:24 server kopano-server[1066]: Error while connecting to search on "file:///var/run/kopano/search.sock" Oct 19 13:26:24 server kopano-server[1066]: message repeated 5 times: [ Error while connecting to search on "file:///var/run/kopano/search.sock"] Oct 19 13:26:24 server apache: [webmail.external.domain] 192.168.1.4 - USER@internal.domain [19/Oct/2017:13:26:24 +0200] "POST /kopano.php?service=fingerprint&type=keepalive HTTP/1.1" 200 - "https://webmail.external.domain/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" Oct 19 13:26:24 server kopano-server[1066]: Error while connecting to search on "file:///var/run/kopano/search.sock" Oct 19 13:26:24 server kopano-server[1066]: Error while connecting to search on "file:///var/run/kopano/search.sock" Oct 19 13:26:24 server apache: [webmail.external.domain] 192.168.1.4 - USER@internal.domain [19/Oct/2017:13:26:24 +0200] "POST /kopano.php?subsystem=webapp_1508412384436 HTTP/1.1" 200 3020 "https://webmail.external.domain/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" Oct 19 13:26:24 server apache: [webmail.external.domain] 192.168.1.4 - USER@internal.domain [19/Oct/2017:13:26:24 +0200] "POST /kopano.php?subsystem=webapp_1508412384436 HTTP/1.1" 200 1247 "https://webmail.external.domain/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" Oct 19 13:26:26 server kopano-server[1066]: Error while connecting to search on "file:///var/run/kopano/search.sock" Oct 19 13:26:26 server kopano-server[1066]: Error while connecting to search on "file:///var/run/kopano/search.sock" Oct 19 13:26:26 server apache: [webmail.external.domain] 192.168.1.4 - USER@internal.domain [19/Oct/2017:13:26:26 +0200] "POST /kopano.php?subsystem=webapp_1508412384436 HTTP/1.1" 200 337 "https://webmail.external.domain/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" Oct 19 13:26:44 server z-push/core[7680]: [INFO][USER] StatusException: SyncCollections->CheckForChanges(): Timeout forced after 42s from 600s due to other process - code: 3 - file: /usr/share/z-push/lib/core/synccollections.php:579 Oct 19 13:26:44 server z-push/core[7680]: [DEBUG][USER] WBXMLEncoder->startWBXML() type: vnd.ms-sync.wbxml Oct 19 13:26:44 server z-push/core[7680]: [WBXML][USER] O <Ping:Ping> Oct 19 13:26:44 server z-push/core[7680]: [WBXML][USER] O <Ping:Status> Oct 19 13:26:44 server z-push/core[7680]: [WBXML][USER] O 1 Oct 19 13:26:44 server z-push/core[7680]: [WBXML][USER] O </Ping:Status> Oct 19 13:26:44 server z-push/core[7680]: [WBXML][USER] O </Ping:Ping> Oct 19 13:26:44 server z-push/core[7680]: [DEBUG][USER] WBXMLEncoder->endTag() WBXML output completed Oct 19 13:26:44 server z-push/core[7680]: [WBXML][USER] WBXML-OUT: AwFqAAANRUcDMQABAQ== Oct 19 13:26:44 server z-push/core[7680]: [WBXML][USER] WBXML-IN : Oct 19 13:26:44 server z-push/core[7680]: [DEBUG][USER] LoopDetection->ProcessLoopDetectionTerminate() Oct 19 13:26:44 server z-push/core[7680]: [INFO][USER] cmd='Ping' memory='2.37 MiB/2.00 MiB' time='43.15s' devType='iPhone' devId='88imeq36lh03d2gbjkh272j5ko' getUser='USER' from='IPADDRESS' idle='42s' version='2.3.8+0-0' method='POST' httpcode='200' Oct 19 13:26:44 server z-push/core[7680]: [DEBUG][USER] -------- End Oct 19 13:26:44 server apache: [activesync.external.domain] IPADDRESS - USER [19/Oct/2017:13:26:01 +0200] "POST /Microsoft-Server-ActiveSync?User=USER&DeviceId=88IMEQ36LH03D2GBJKH272J5KO&DeviceType=iPhone&Cmd=Ping HTTP/1.1" 200 13 "-" "Apple-iPhone9C3/1501.402" Oct 19 13:26:54 server apache: [webmail.external.domain] 192.168.1.4 - USER@internal.domain [19/Oct/2017:13:26:54 +0200] "POST /kopano.php?subsystem=webapp_1508412384436 HTTP/1.1" 200 113 "https://webmail.external.domain/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063" Oct 19 13:26:55 server apache: [Thu Oct 19 13:26:55.867922 2017] [core:notice] [pid 7670] AH00051: child pid 7679 exit signal Segmentation fault (11), possible coredump in /etc/apache2
I just recognized that there is a problem with kopano-search and kopano-presence too.
Regards
Richard -
HI @darootler ,
the segfault is logged 11 seconds after the last z-push log message. are you certain that the segfault and z-push are related?
You could disable the z-push vhost to rule this one out (and afterwards do the same with the webapp).
ps: if the log size is a problem you could still uploade it for example to https://pastebin.com/
-
Hi @fbartels
no i am not sure that the segfault is related to z-push. It was just a guess. I also don’t know what “libkcclient.so” is for, that’s why i opened a thread here.
Well i don’t want to disable z-push because i am using it.
Regards
Richard -
Hi @fbartels,
i disabled the kopano webapp vhost because i am using it very rarely.
I‘ll report back.
Regards
Richard -
Hi @fbartels,
i just received another segfault from apache. So in my opinion this error is related to z-push because the webapp vhost was disabled.
Regards
Richard -
I would say you should generate a coredump and send it to Kopano.
The linked ticket is old, fixed & closed and probably not very helpful. -
@sebastian said in libkcclient.so causing Apache Segmentation fault:
I would say you should generate a coredump and send it to Kopano.
The linked ticket is old, fixed & closed and probably not very helpful.Okay, how do send it to Kopano? Mail? Ticket?
Regards
Richard -
I am not receiving this error since uprading to version 8.4.90.550.
Regards
Richard