Segmentation Fault upon fresh installation

ivanlaszik

Long time zarafa user in home environment.
Now I want to build a fresh new server arround Fedora 26 and am starting with Kopano.
Downloaded build core-8.4.90.1338_0+158-Fedora_26-x86_64 on Dec. 12.
Fresh, out of the box, fedora 26 installation, installed only mariadb, httpd and php: #dnf install mariadb-server mariadb-devel httpd php. Then cd into folder core-8.4.90.1338_0+158-Fedora_26-x86_64 and ran: #dnf localinstall *. it presented the install + dependencies, typed y and everything installed succesfully.
Created the database and user in mysql, and started Kopano. No issues until here, mysql tables were created.
As soon as I issued #kopano-admin … to create my first admin user, the output was:

kopano-admin -c kopano -P -e kopano@localdomain.com -f "Kopano Admin" -a 1
Segmentation fault (core dumped)

In /var/log/messages I get:

Dec 13 18:48:48 kopanosrv kernel: kopano-admin[7120]: segfault at 4 ip 00007f0fa3f163b4 sp 00007ffd61ab3560 error 4 in libcrypto.so.1.0.2m[7f0fa3dac000+236000]
Dec 13 18:48:48 kopanosrv audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@10-7124-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 13 18:48:48 kopanosrv systemd[1]: Started Process Core Dump (PID 7124/UID 0).
Dec 13 18:48:48 kopanosrv systemd-coredump[7125]: Process 7120 (kopano-admin) of user 0 dumped core.#012#012Stack trace of thread 7120:#012#0  0x00007f0fa3f163b4 x509_verify_param_zero (libcrypto.so.10)#012#1  0x00007f0fa3f1651e X509_VERIFY_PARAM_free (libcrypto.so.10)#012#2  0x00007f0fa424fe81 SSL_CTX_free (libssl.so.10)#012#3  0x00007f0fa4dfe639 soap_done (libgsoapssl++-kopano-2.8.54.so)#012#4  0x00007f0fa4e17eb9 soap_free (libgsoapssl++-kopano-2.8.54.so)#012#5  0x00007f0fa588c575 _ZN4KCmdD4Ev (libkcclient.so)#012#6  0x00007f0fa586e300 _ZN11WSTransport8HrLogOffEv (libkcclient.so)#012#7  0x00007f0fa586e5ca _ZN11WSTransportD1Ev (libkcclient.so)#012#8  0x00007f0fa586e7a9 _ZN11WSTransportD0Ev (libkcclient.so)#012#9  0x00007f0fb0d8cb6b _ZN2KC9ECUnknown7SuicideEv (libkcmapi.so.0)#012#10 0x00007f0fb0d8cf2f _ZN2KC9ECUnknown7ReleaseEv (libkcmapi.so.0)#012#11 0x00007f0fa588b051 _ZN4KCHL10object_ptrI11WSTransportED4Ev (libkcclient.so)#012#12 0x00007f0fb0fd4826 _ZN18M4LMsgServiceAdmin19ConfigureMsgServiceEPK7MAPIUIDmjjPK10SPropValue (libmapi.so.1)#012#13 0x00007f0fb0fe5191 CreateProfileTemp (libmapi.so.1)#012#14 0x00007f0fb0fe5596 _ZN2KC15HrOpenECSessionEPP12IMAPISessionPKcS4_PKwS6_S4_jS4_S4_S4_ (libmapi.so.1)#012#15 0x00007f0fb0fe5790 _ZN2KC20HrOpenECAdminSessionEPP12IMAPISessionPKcS4_S4_jS4_S4_ (libmapi.so.1)#012#16 0x000000000040952d main (kopano-admin)#012#17 0x00007f0fac8ae88a __libc_start_main (libc.so.6)#012#18 0x000000000041826a _start (kopano-admin)
Dec 13 18:48:48 kopanosrv audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-coredump@10-7124-0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 13 18:48:49 kopanosrv gdb[7151]: detected unhandled Python exception in 'interactive mode'
Dec 13 18:48:49 kopanosrv abrt-server[7129]: Deleting problem directory ccpp-2017-12-13-18:48:48.846728-7120 (dup of ccpp-2017-12-12-22:22:43.108924-2163)
Dec 13 18:48:49 kopanosrv dbus-daemon[549]: [system] Activating service name='org.freedesktop.problems' requested by ':1.54' (uid=0 pid=7168 comm="/usr/bin/python3 /usr/bin/abrt-action-notify -d /v") (using servicehelper)
Dec 13 18:48:49 kopanosrv dbus-daemon[549]: [system] Successfully activated service 'org.freedesktop.problems'
Dec 13 18:48:49 kopanosrv abrt-notification[7174]: Process 2163 (kopano-admin) crashed in x509_verify_param_zero()

At first I thought its the builds fault so I waited for the next one. Today I removed it and installed core-8.4.90.1346_0+159-Fedora_26-x86_64 but the same is happening.

kopano-admin -V, or without any parameters, or other kopano- commands with -V return the version number.

I also installed webapp via dnf localinstall *, got no errors in the installation process.
The login page comes up, but upon entering any user + password and hitting sign in, “This page isn’t working” is returned in Chrome, and in /var/log/messages I get:

Dec 13 19:11:29 kopanosrv audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 13 19:12:00 kopanosrv audit[7563]: ANOM_ABEND auid=4294967295 uid=48 gid=48 ses=4294967295 pid=7563 comm="httpd" exe="/usr/sbin/httpd" sig=11 res=1
Dec 13 19:12:00 kopanosrv audit[7605]: ANOM_ABEND auid=4294967295 uid=48 gid=48 ses=4294967295 pid=7605 comm="httpd" exe="/usr/sbin/httpd" sig=11 res=1

Kopano server.log, however, recognizes the login atempt:

Wed Dec 13 19:12:00 2017: [warning] Authentication by plugin failed for user "kopano": Trying to authenticate failed: wrong username or password

Any help is appreciated. Let me know what other info I should provide.

Ivan.

lherrmann

Hi,

Thanks for the report!
I tried to reproduce this, but unfortunately I couldn’t do this with Fedora, so I tested it with centos/7 (version 8.4.90.1362). With this environment there was no problem with adding the admin user.

ivanlaszik

Thanks for the reply, lherrmann.
So thats it, if no repro possible, no help?
Anyone else? at least what else to do for troubleshooting?

bosim

@ivanlaszik said in Segmentation Fault upon fresh installation:

Anyone else? at least what else to do for troubleshooting?

To me it looks that you have libressl installed? Please try with openssl only. I got these kind of crashes because I had both libressl and openssl installed.

Bo

ivanlaszik

Thanks bosim.
I checked what ssl i have installed:

[root@kopanosrv ~]# rpm -qa | grep ssl
libkcssl0-8.4.90.1346-159.1.x86_64
compat-openssl10-1.0.2m-1.fc26.x86_64
openssl-libs-1.1.0g-1.fc26.x86_64
openssl-devel-1.1.0g-1.fc26.x86_64
openssl-1.1.0g-1.fc26.x86_64
xmlsec1-openssl-1.2.23-2.fc26.x86_64

There is no libressl installed.

bosim

@ivanlaszik said in Segmentation Fault upon fresh installation:

There is no libressl installed.

Try loooking for:

libcrypto.so.10

Bo

ivanlaszik

I have libcrypto.so.10 from compat-openssl:

[root@kopanosrv ~]# rpm -ql compat-openssl10
/usr/lib64/.libcrypto.so.1.0.2m.hmac
/usr/lib64/.libcrypto.so.10.hmac
/usr/lib64/.libssl.so.1.0.2m.hmac
/usr/lib64/.libssl.so.10.hmac
/usr/lib64/libcrypto.so.1.0.2m
/usr/lib64/libcrypto.so.10
/usr/lib64/libssl.so.1.0.2m
/usr/lib64/libssl.so.10

however this was installed by DNF as a dependency when I installed the kopano packages:

[root@kopanosrv ~]# dnf history info 5 | grep compat
Command Line   : localinstall gsoap-kopano-debuginfo-2.8.54-1.1.x86_64.rpm gsoap-kopano-devel-2.8.54-1.1.x86_64.rpm gsoap-kopano-doc-2.8.54-1.1.noarch.rpm kopano-backup-8.4.90.1338-158.1.x86_64.rpm kopano-bash-completion-8.4.90.1338-158.1.noarch.rpm kopano-client-8.4.90.1338-158.1.x86_64.rpm kopano-common-8.4.90.1338-158.1.x86_64.rpm kopano-compat-8.4.90.1338-158.1.x86_64.rpm kopano-contacts-8.4.90.1338-158.1.x86_64.rpm kopano-dagent-8.4.90.1338-158.1.x86_64.rpm kopano-debuginfo-8.4.90.1338-158.1.x86_64.rpm kopano-devel-8.4.90.1338-158.1.x86_64.rpm kopano-gateway-8.4.90.1338-158.1.x86_64.rpm kopano-ical-8.4.90.1338-158.1.x86_64.rpm kopano-lang-8.4.90.1338-158.1.x86_64.rpm kopano-migration-imap-8.4.90.1338-158.1.noarch.rpm kopano-migration-pst-8.4.90.1338-158.1.x86_64.rpm kopano-monitor-8.4.90.1338-158.1.x86_64.rpm kopano-presence-8.4.90.1338-158.1.x86_64.rpm kopano-search-8.4.90.1338-158.1.x86_64.rpm kopano-server-8.4.90.1338-158.1.x86_64.rpm kopano-server-packages-8.4.90.1338-158.1.x86_64.rpm kopano-spooler-8.4.90.1338-158.1.x86_64.rpm kopano-utils-8.4.90.1338-158.1.x86_64.rpm libgsoap-kopano-2_8_54-2.8.54-1.1.x86_64.rpm libkcarchiver0-8.4.90.1338-158.1.x86_64.rpm libkcarchivercore0-8.4.90.1338-158.1.x86_64.rpm libkcfreebusy0-8.4.90.1338-158.1.x86_64.rpm libkcicalmapi0-8.4.90.1338-158.1.x86_64.rpm libkcinetmapi0-8.4.90.1338-158.1.x86_64.rpm libkcmapi0-8.4.90.1338-158.1.x86_64.rpm libkcserver0-8.4.90.1338-158.1.x86_64.rpm libkcsoap0-8.4.90.1338-158.1.x86_64.rpm libkcssl0-8.4.90.1338-158.1.x86_64.rpm libkcsync0-8.4.90.1338-158.1.x86_64.rpm libkcutil0-8.4.90.1338-158.1.x86_64.rpm libmapi1-8.4.90.1338-158.1.x86_64.rpm libs3-4-4.1.git257-2.1.x86_64.rpm libs3-debuginfo-4.1.git257-2.1.x86_64.rpm libs3-devel-4.1.git257-2.1.x86_64.rpm libs3-tools-4.1.git257-2.1.x86_64.rpm libtidy5-5.1.9-1.3.x86_64.rpm libvmime-devel-0.9.2-13.1.x86_64.rpm libvmime-kopano1-0.9.2-13.1.x86_64.rpm libvmime_kopano-debuginfo-0.9.2-13.1.x86_64.rpm php-mapi-8.4.90.1338-158.1.x86_64.rpm python-kopano-8.4.90.1338-158.1.x86_64.rpm python-mapi-8.4.90.1338-158.1.x86_64.rpm tidy-html5-5.1.9-1.3.x86_64.rpm tidy-html5-debuginfo-5.1.9-1.3.x86_64.rpm tidy-html5-devel-5.1.9-1.3.x86_64.rpm
  Install compat-openssl10-1:1.0.2m-1.fc26.x86_64            @updates
  Install kopano-compat-8.4.90.1338-158.1.x86_64             @@commandline

any advice?

bosim

@ivanlaszik said in Segmentation Fault upon fresh installation:

any advice?

What happens if you try removing the package?

Bo

ivanlaszik

I did that, by #dnf remove compat-openssl10, and it offered to remove 64 packages, among them all the kopano packages.

fbartels

@ivanlaszik said in Segmentation Fault upon fresh installation:

what else to do for troubleshooting?

I’d recommend to stick with a distribution that is also used for the supported packages (see https://documentation.kopano.io/kopanocore_administrator_manual/installing.html#supported-platforms for an overview). The listed distributions are tested on a regular base and should therefore (theoretically) be free of such errors.

ivanlaszik

Thanks fbartels. My current working (old) zarafa installation is on Fedora 22 and there were no issue from the beginning.
For personal use I love to play with cutting edge stuff, reason why I choose Fedora.
I am sure it will work with CentOS7, with which I will go if I cannot make it on FC, but in the meantime I really want to work it out with FC26.

hodor27

I also get a sig-segfault exception on Ubuntu 16.04.3 LTS upon running kopano-admin -l

/var/log/syslog

Dec 20 17:31:01 mail kernel: [  143.005424] kopano-admin[7585]: segfault at a ip 000000000000000a sp 00007ffd21d4bb98 error 14 in kopano-admin[400000+29000]

also i found this in /var/log/kopano/server.log

Wed Dec 20 17:31:38 2017: [crit   ] ----------------------------------------------------------------------
Wed Dec 20 17:31:38 2017: [crit   ] Fatal error detected. Please report all following information.
Wed Dec 20 17:31:38 2017: [crit   ] Application kopano-server version: 8.4.5.0
Wed Dec 20 17:31:38 2017: [crit   ] OS: Linux, release: 4.4.0-103-generic, version: #126-Ubuntu SMP Mon Dec 4 16:23:28 UTC 2017, hardware: x86_64
Wed Dec 20 17:31:38 2017: [crit   ] Thread name: NotificationMan
Wed Dec 20 17:31:38 2017: [crit   ] Peak RSS: 16920
Wed Dec 20 17:31:38 2017: [crit   ] Pid 7462 caught SIGSEGV (11), traceback:
Wed Dec 20 17:31:38 2017: [crit   ] Backtrace:

when i try get a backtrace for kopano-admin:

root@mail:/etc/kopano# gdb kopano-admin 
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from kopano-admin...(no debugging symbols found)...done.
(gdb) run -l
Starting program: /usr/sbin/kopano-admin -l
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00000000ffffffff in ?? ()
(gdb) bt
#0  0x00000000ffffffff in ?? ()
#1  0x00007ffff2681653 in soap_delete () from /usr/lib/libgsoapssl++-kopano-2.8.57.so
#2  0x00007ffff2f02bb3 in ?? () from /usr/lib/kopano/libkcclient.so
#3  0x00007ffff2ee1053 in ?? () from /usr/lib/kopano/libkcclient.so
#4  0x00007ffff2ee1add in ?? () from /usr/lib/kopano/libkcclient.so
#5  0x00007ffff2f01be3 in MSGServiceEntry () from /usr/lib/kopano/libkcclient.so
#6  0x00007ffff7b8362c in ?? () from /usr/lib/libmapi.so.1
#7  0x00007ffff7b91cd1 in ?? () from /usr/lib/libmapi.so.1
#8  0x00007ffff7b92e26 in KC::HrOpenECSession(IMAPISession**, char const*, char const*, wchar_t const*, wchar_t const*, char const*, unsigned int, char const*, char const*, char const*) () from /usr/lib/libmapi.so.1
#9  0x00007ffff7b93020 in KC::HrOpenECAdminSession(IMAPISession**, char const*, char const*, char const*, unsigned int, char const*, char const*) () from /usr/lib/libmapi.so.1
#10 0x00000000004090f0 in ?? ()
#11 0x00007ffff694e830 in __libc_start_main (main=0x4063a0, argc=2, argv=0x7fffffffe708, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe6f8) at ../csu/libc-start.c:291
#12 0x0000000000417769 in ?? ()

hodor27

upon a fresh install of kopano the first segfault i get is from spooler:

Dec 20 18:02:30 mail systemd[1]: kopano-spooler.service: Main process exited, code=killed, status=11/SEGV
Dec 20 18:02:30 mail systemd[1]: kopano-spooler.service: Unit entered failed state.
Dec 20 18:02:30 mail systemd[1]: kopano-spooler.service: Failed with result 'signal'.
Dec 20 18:02:30 mail kernel: kopano-spooler[15267]: segfault at a ip 00007f92c8792ccb sp 00000000025c8820 error 4 in libgcc_s.so.1[7f92c8784000+16000]
Dec 20 18:02:30 mail kernel: kopano-admin[15306]: segfault at a ip 000000000000000a sp 00007ffc51b40c28 error 14 in kopano-admin[400000+29000]

hodor27

the backtace for kopano-spooler looks similiar to the one kopano-admin produces:

root@mail:/etc/kopano# gdb /usr/sbin/kopano-spooler 
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/kopano-spooler...(no debugging symbols found)...done.
(gdb) run -F
Starting program: /usr/sbin/kopano-spooler -F
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffedf40700 (LWP 15581)]

Thread 1 "kopano-spooler" received signal SIGSEGV, Segmentation fault.
0x00000000ffffffff in ?? ()
(gdb) bt
#0  0x00000000ffffffff in ?? ()
#1  0x00007fffef594653 in soap_delete () from /usr/lib/libgsoapssl++-kopano-2.8.57.so
#2  0x00007ffff001ebb3 in ?? () from /usr/lib/kopano/libkcclient.so
#3  0x00007fffefffd053 in ?? () from /usr/lib/kopano/libkcclient.so
#4  0x00007fffefffdadd in ?? () from /usr/lib/kopano/libkcclient.so
#5  0x00007ffff001dbe3 in MSGServiceEntry () from /usr/lib/kopano/libkcclient.so
#6  0x00007ffff790362c in ?? () from /usr/lib/libmapi.so.1
#7  0x00007ffff7911cd1 in ?? () from /usr/lib/libmapi.so.1
#8  0x00007ffff7912e26 in KC::HrOpenECSession(IMAPISession**, char const*, char const*, wchar_t const*, wchar_t const*, char const*, unsigned int, char const*, char const*, char const*) () from /usr/lib/libmapi.so.1
#9  0x00007ffff7913020 in KC::HrOpenECAdminSession(IMAPISession**, char const*, char const*, char const*, unsigned int, char const*, char const*) () from /usr/lib/libmapi.so.1
#10 0x000000000040a2eb in ?? ()
#11 0x00007ffff5ef7830 in __libc_start_main (main=0x409450, argc=2, argv=0x7fffffffe708, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe6f8) at ../csu/libc-start.c:291
#12 0x000000000040b439 in ?? ()

fbartels

Hi @hodor27 ,

Can you post the output of dpkg -l?

hodor27

@fbartels

dpkg -l output is pastede here: https://pastebin.com/N5euKyQi

@ivanlaszik said in Segmentation Fault upon fresh installation:

Downloaded build core-8.4.90.1338_0+158-Fedora_26-x86_64 on Dec. 12.

F26 build now (kopano-8.4.90.1513-170.1) uses openssl 1.0 throughout. Maybe that cures the issue.

ivanlaszik

installed core-8.4.90.1513_0+172-Fedora_26-x86_64 just now and creation of the admin user was succesfull.
upgraded webapp to 3.4.3.1127+669 also, however, i get following error in httpd error log upon login attempt:

[Tue Dec 26 17:19:02.577640 2017] [php7:error] [pid 2078] [client x.x.x.x:59313] PHP Fatal error:  Uncaught Error: Class 'SimpleXMLElement' not found in /usr/share/kopano-webapp/server/includes/core/class.pluginmanager.php:849\nStack trace:\n#0 /usr/share/kopano-webapp/server/includes/core/class.pluginmanager.php(417): PluginManager->extractPluginDataFromXML('<?xml version="...', 'desktopnotifica...')\n#1 /usr/share/kopano-webapp/server/includes/core/class.pluginmanager.php(211): PluginManager->processPlugin('desktopnotifica...')\n#2 /usr/share/kopano-webapp/server/includes/core/class.pluginmanager.php(164): PluginManager->readPluginFolder(Array)\n#3 /usr/share/kopano-webapp/index.php(167): PluginManager->detectPlugins('')\n#4 {main}\n  thrown in /usr/share/kopano-webapp/server/includes/core/class.pluginmanager.php on line 849, referer: http://x.x.x.x/webapp/?logon

and browser shows:

x.x.x.x is currently unable to handle this request.
HTTP ERROR 500

I don’t think this is related to the previous problem.
i am researching, but any advice is appreciated.

ivanlaszik

fixed the above by installing php-xml.

all is good now, kopano seems to be working.

thanks for all the help and input :).