SerializeObject failed with error code 0x80000004 for object

martimcfly

Hej jengelh!

only the client dies (search, backup… never tried archive or others). kopano-server, mysql, webapp do well.

We can do gdb through Teamviewer. With some advice about the needed build flags I can prepare the server.

Due to personal data and size I’m not able to handout the database dump. But I extracted the properties for a failed message. Its only a newsletter but still containing e-mail-adresses. I can show that to you by teamviewer :)

Would teamviewer be an option or would you preferre other options?

Thanks so much for your effort so far!

Marti

jengelh

TV will do. Given that TeamViewer Linux client has had its integrated chat functionality gutted from v13, I will point out that irc.freenode.net/#kopano is a possible alternative for textual communication during the session.

martimcfly

@jengelh Can you tell me the debug build command in advanced please? I build this for Armv7 and it takes a while to finish. Or don’t you need all symbols for gdb?

jengelh

(assuming Debian 9/10) apt-get install autoconf automake autotools-dev binutils debhelper devscripts fakeroot g++ gettext gsoap libcurl4-openssl-dev libical-dev libicu-dev libjsoncpp-dev libkrb5-dev libldap2-dev libmariadbclient-dev libncurses5-dev libncursesw5-dev libpam0g-dev libpopt-dev libssl-dev libtool libtool-bin libvmime-dev libxml2-dev m4 openssl php-dev pkg-config python3-dev python3-setuptools python-dev python-setuptools swig uuid-dev zlib1g-dev git-core gdb python-nose python-vobject joe valgrind python-requests

git clone https://github.com/kopano-dev/kopano-core

./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var CXXFLAGS="-O0 -ggdb3"

martimcfly

Well, I compiled it with those flags. It went all through and the package have gotten bigger. But it appears as if the symbols haven’t been installed with “make install”.

Following is giving empty output.

readelf -s /usr/lib/libkcserver.so | grep Serialize


Install prefix	:	/
Userscript dir	:	/etc/kopano/userscripts
Quota dir	:	/etc/kopano/quotamails
Search scripts dir	:	/etc/kopano/searchscripts

Build host	:	armv7l-unknown-linux-gnueabihf
Compile flags	:	 -DLINUX 	-I${top_builddir}/common/include 	-I${top_srcdir}/common/include 	-I${top_srcdir}/m4lcommon/include 	-I${top_srcdir}/mapi4linux/include 	-I${top_srcdir}/provider/common/include 	-I${top_srcdir}/provider/libserver/include 	-I${top_srcdir}/ECtools/archiver/include 	-I${top_srcdir}/inetmapi/include 	-I${top_srcdir}/swig/include 	-I${top_srcdir}/swig/python/include 	-I${top_srcdir} -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG -DUNICODE  -DUSE_EPOLL   -I/usr/include/python2.7 -Wall -Wmissing-declarations -std=c++11 -Wno-variadic-macros -O2 -fno-strict-aliasing -fvisibility=hidden -fvisibility-inlines-hidden -O0 -ggdb3
Debug flags	:	
PHP version	:	7.2.3
PHP ext dir	:	/usr/lib/php/modules
PHP config dir	:	/etc/php/conf.d
PHP flags	:	-I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib
PHP libs	:	
gSOAP flags	:	-DWITH_OPENSSL -DWITH_GZIP -DHAVE_OPENSSL_SSL_H -DHAVE_ZLIB_H
gSOAP libs	:	-lgsoapssl++
gSOAP soapcpp2	:	
OpenSSL libs	:	-lssl
MySQL flags	:	-I/usr/include/mysql -I/usr/include/mysql/..
MySQL libs	:	-L/usr/lib  -lmysqlclient -lpthread -lz -lm -ldl -lssl -lcrypto
VMime flags	:	-I/usr/include/vmime/ -I/usr/include/p11-kit-1
VMime libs	:	-lvmime
XML2 flags	:	-I/usr/include/libxml2
XML2 libs	:	-lxml2
ICal flags	:	
ICal libs	:	-lical -licalss -licalvcal -lpthread /lib/libicuuc.so /lib/libicui18n.so
ncurses libs	:	-lncursesw
Clients libs	:	-luuid   -lz
LDAP support	:	yes
LDAP flags	:	-DLDAP_DEPRECATED=1
LDAP libs	:	-lldap_r
Curl libs	:	-lcurl
Pam libs	:	-lpam
Krb5 libs	:	-lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err
TCMalloc lib	:	-ltcmalloc_minimal
S3 lib		:	
KCOIDC lib	:	
ICU libs	:	-licuuc -licudata -licui18n -licuuc -licudata
Python flags	:	-I/usr/include/python2.7
Python libs	:	-lpython2.7

...

libtool: install: /usr/bin/install -c .libs/libkcsoap.so.0.0.0T /build/kopano-core/pkg/kopano-core/usr/lib/libkcsoap.so.0.0.0
libtool: install: (cd /build/kopano-core/pkg/kopano-core/usr/lib && { ln -s -f libkcsoap.so.0.0.0 libkcsoap.so.0 || { rm -f libkcsoap.so.0 && ln -s libkcsoap.so.0.0.0 libkcsoap.so.0; }; })
libtool: install: (cd /build/kopano-core/pkg/kopano-core/usr/lib && { ln -s -f libkcsoap.so.0.0.0 libkcsoap.so || { rm -f libkcsoap.so && ln -s libkcsoap.so.0.0.0 libkcsoap.so; }; })
libtool: install: /usr/bin/install -c .libs/libkcsoap.lai /build/kopano-core/pkg/kopano-core/usr/lib/libkcsoap.la
libtool: warning: relinking 'libkcserver.la'
libtool: install: (cd /build/kopano-core/src/kopanocore; /bin/sh "/build/kopano-core/src/kopanocore/libtool"  --silent --tag CXX --mode=relink g++ -Wall -Wmissing-declarations -std=c++11 -Wno-variadic-macros -O2 -fno-strict-aliasing -fvisibility=hidden -fvisibility-inlines-hidden -Wno-sign-compare -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector-strong -fno-plt -Wl,--version-script=provider/libkcserver.sym -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o libkcserver.la -rpath /usr/lib common/database.lo provider/libserver/ECABObjectTable.lo provider/libserver/ECAttachmentStorage.lo provider/libserver/ECCacheManager.lo provider/libserver/ECConvenientDepthObjectTable.lo provider/libserver/ECDatabaseFactory.lo provider/libserver/ECDatabaseMySQL.lo provider/libserver/ECDatabaseUpdate.lo provider/libserver/ECDatabaseUtils.lo provider/libserver/ECGenProps.lo provider/libserver/ECGenericObjectTable.lo provider/libserver/ECICS.lo provider/libserver/ECICSHelpers.lo provider/libserver/ECIndexer.lo provider/libserver/ECKrbAuth.lo provider/libserver/ECLockManager.lo provider/libserver/ECMailBoxTable.lo provider/libserver/ECMultiStoreTable.lo provider/libserver/ECNotificationManager.lo provider/libserver/ECPluginFactory.lo provider/libserver/ECPluginSharedData.lo provider/libserver/ECS3Attachment.lo provider/libserver/ECSearchFolders.lo provider/libserver/ECSearchObjectTable.lo provider/libserver/ECSecurity.lo provider/libserver/ECServerEntrypoint.lo provider/libserver/ECSession.lo provider/libserver/ECSessionGroup.lo provider/libserver/ECSessionManager.lo provider/libserver/ECStatsCollector.lo provider/libserver/ECStatsTables.lo provider/libserver/ECStoreObjectTable.lo provider/libserver/ECStringCompat.lo provider/libserver/ECSubRestriction.lo provider/libserver/ECTPropsPurge.lo provider/libserver/ECTableManager.lo provider/libserver/ECTestProtocol.lo provider/libserver/ECUserManagement.lo provider/libserver/ECUserStoreTable.lo provider/libserver/StorageUtil.lo provider/libserver/StreamUtil.lo provider/libserver/cmd.lo provider/libserver/cmdutil.lo provider/libserver/logontime.lo provider/common/ECSearchClient.lo provider/common/SOAPDebug.lo libkcutil.la libkcmapi.la libkcssl.la libkcsoap.la -lpthread -licui18n -licuuc -licudata -licuuc -licudata -lgsoapssl++ -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -L/usr/lib -lmysqlclient -lpthread -lz -lm -ldl -lssl -lcrypto -lpam -lssl -inst-prefix-dir /build/kopano-core/pkg/kopano-core)
libtool: install: /usr/bin/install -c .libs/libkcserver.so.0.0.0T /build/kopano-core/pkg/kopano-core/usr/lib/libkcserver.so.0.0.0
libtool: install: (cd /build/kopano-core/pkg/kopano-core/usr/lib && { ln -s -f libkcserver.so.0.0.0 libkcserver.so.0 || { rm -f libkcserver.so.0 && ln -s libkcserver.so.0.0.0 libkcserver.so.0; }; })
libtool: install: (cd /build/kopano-core/pkg/kopano-core/usr/lib && { ln -s -f libkcserver.so.0.0.0 libkcserver.so || { rm -f libkcserver.so && ln -s libkcserver.so.0.0.0 libkcserver.so; }; })
libtool: install: /usr/bin/install -c .libs/libkcserver.lai /build/kopano-core/pkg/kopano-core/usr/lib/libkcserver.la

Marti

jengelh

that’s because readelf for some stupid reason truncates the names by default, even if they’re in the last column (one explicitly needs -W), so a trivial grep would not find it. nm is better suited for that.

martimcfly

Badly not…

[root@server lib]# nm libkcserver.so
nm: libkcserver.so: no symbols

[root@serverlib lib]# readelf -w libkcserver.so
Contents of the .eh_frame section:

00000000 ZERO terminator


[root@server lib]#

It didn’t matter if I did this…

./bootstrap.sh
CPPFLAGS="-I/usr/include/python2.7" CXXFLAGS="-O0 -ggdb3" ./configure \
  --prefix=/ \
  --localstatedir=/var \
  --sysconfdir=/etc \
  --exec-prefix=/usr \
  --sbindir=/usr/bin \
  --datarootdir=/usr/share \
  --includedir=/usr/include \
  --enable-release \
  --enable-epoll \
  --enable-python \
  --disable-debug \
  --disable-static \
  --with-userscript-prefix=/etc/kopano/userscripts \
  --with-quotatemplate-prefix=/etc/kopano/quotamails \
  --with-searchscripts-prefix=/etc/kopano/searchscripts \
  --with-php=7 \
  PYTHON=/usr/bin/python2

make
make install DESTDIR="$(realpath ${pkgdir})"

or this

./bootstrap.sh
CPPFLAGS="-I/usr/include/python2.7" ./configure \
 CXXFLAGS="-O0 -ggdb3" \
  --prefix=/ \
  --localstatedir=/var \
  --sysconfdir=/etc \
  --exec-prefix=/usr \
  --sbindir=/usr/bin \
  --datarootdir=/usr/share \
  --includedir=/usr/include \
  --enable-release \
  --enable-epoll \
  --enable-python \
  --disable-debug \
  --disable-static \
  --with-userscript-prefix=/etc/kopano/userscripts \
  --with-quotatemplate-prefix=/etc/kopano/quotamails \
  --with-searchscripts-prefix=/etc/kopano/searchscripts \
  --with-php=7 \
  PYTHON=/usr/bin/python2

make
make install DESTDIR="$(realpath ${pkgdir})"

Here is the log for the last build command…

https://gist.githubusercontent.com/mpietruschka/79ef730d2fa2defeac80a0301d59e3ec/raw/a911d8dfe68fa8a477de54dd681d1b39804a5f9e/build_kopano-core.log

This is the packaging script…

https://gist.githubusercontent.com/mpietruschka/8c8293b1898e9de53bbe77f0f50607f1/raw/ee256e7a9d26c7d70a0e39d1483dcb024e1df46c/PKGBUILD

Any ideas?

jengelh

==> Tidying install...
  -> Removing libtool files...
  -> Purging unwanted files...
  -> Removing static library files...
  -> Stripping unneeded symbols from binaries and libraries...

This strip may be it, and it’s not coming from KC. But worry not, kopano-server can be started right from the build directory, it does not need make install or extra packaging for a debug session.

nm .libs/libkcserver.so

martimcfly

You’re awesome! This has been it!

I had to change the packager defaults to not strip symbols or remove libtool .la files and debug informations. I disabled it specifically for this package.

/etc/makepkg.conf

#########################################################################
# GLOBAL PACKAGE OPTIONS
#   These are default values for the options=() settings
#########################################################################
#
# Default: OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !optipng !upx !debug)
#  A negated option will do the opposite of the comments below.
#
#-- strip:      Strip symbols from binaries/libraries
#-- docs:       Save doc directories specified by DOC_DIRS
#-- libtool:    Leave libtool (.la) files in packages
#-- staticlibs: Leave static library (.a) files in packages
#-- emptydirs:  Leave empty directories in packages
#-- zipman:     Compress manual (man and info) pages in MAN_DIRS with gzip
#-- purge:      Remove files specified by PURGE_TARGETS
#-- upx:        Compress binary executable files using UPX
#-- optipng:    Optimize PNG images with optipng
#-- debug:      Add debugging flags as specified in DEBUG_* variables
#
OPTIONS=(!strip docs libtool staticlibs emptydirs zipman purge !optipng !upx debug)

The method finally got visible!

[root@server alarm]# nm /usr/lib/libkcserver.so | grep SerializeM
0018da7c t _ZN2KC16SerializeMessageEPNS_9ECSessionEPNS_10ECDatabaseEPNS_19ECAttachmentStorageEPKNS_10StreamCapsEjjjP4GUIDjPNS_12ECSerializerEb

Now we can go on with gdb :D

Marti

martimcfly

@jengelh I couldn’t start kopano from build directory because its build in a chroot environment on a different device :)

jengelh

Let me put up an obligatory questioning of the motive here ;-)

thought you wanted a native build because “I build this for Armv7 and it takes a while to finish”
if you have a fast machine to compile with, the DB could be transferred

I built KC in reasonable time even with an AMD AXP from 2003, which is probably a reasonable assumption for what a today’s raspberry pi ARM would be capable of.

martimcfly

I build for odroid on another ordroid :) both are armv7 and way faster as a RPI. The setup for the chroot environment and compilation take about 30 minutes together. A transfer of the database takes for ever. I don‘t think this is an option ;)

I could speed up compilation with distributed compilation. But I had no time to update my workers with gcc 7.

jengelh

It is because libtidy crashes, and that kills kopano-backup’s worker subprocess, which the main process also completely fails to act on.

The server side then reports 80000004 because the client disappeared in the middle of a send(2) call.

martimcfly

Hey jengelh,

I’d like to say thank you for all the tremendous help you gave going through the code finding the problem in the last place one would expect!

Marti

jengelh

https://github.com/htacg/tidy-html5/issues/739