SSL negotiation failures with TLSv1 and TLSv1.3 against gateway/ical on Debian 10

kuser1

Same issue since upgrading from 8.7.x to core-9.0.2.136.c285120-Debian_9.0-amd64

2020-01-06T10:13:07.121642: [=======] Starting kopano-gateway version 9.0.2 (pid 23159 uid 998)
2020-01-06T10:13:07.121688: [info   ] Coredump status left at system default.
2020-01-06T10:13:07.121804: [info   ] Re-using fd 4 for 0.0.0.0%lo:110
2020-01-06T10:13:07.121828: [info   ] Re-using fd 5 for [::]%lo:110
2020-01-06T10:13:07.121912: [info   ] Re-using fd 6 for 0.0.0.0%lo:143
2020-01-06T10:13:07.121943: [info   ] Re-using fd 7 for [::]%lo:143
2020-01-06T10:13:07.122039: [info   ] Listening on 0.0.0.0:993 (fd 9)
2020-01-06T10:13:07.122094: [info   ] Listening on [::]:993 (fd 10)
2020-01-06T10:13:07.124085: [error  ] Error loading SSL context, POP3S and IMAPS will be disabled
2020-01-06T10:13:07.134085: [info   ] Logger process started on pid 23165

Same SSL Key Files for every Kopano-Component, only Gateway is complaning
Checked permissions, changed every possible config parameter with no success…

thctlo

If you kopano-gateway is not starting at all.

change :

run_as_user = kopano
run_as_group = kopano

to

run_as_user = root
run_as_group = root

Still no SSL, but at least gateway now starts.

kuser1

@thctlo said in SSL negotiation failures with TLSv1 and TLSv1.3 against gateway/ical on Debian 10:

If you kopano-gateway is not starting at all.

change :
run_as_user = kopano
run_as_group = kopano
to
run_as_user = root
run_as_group = root
Still no SSL, but at least gateway now starts.

Yes, the “K-1559: bind [::]:995: Permission denied” goes away by either running as root or executing

setcap cap_net_bind_service=+ep /usr/sbin/kopano-gateway

modelnine

@fbartels I’ve created a PR for this patch on GitHub; for my development, I’m not working with github, so I needed to set up the corresponding repository first. Anyway, independent of the actual changes for OpenSSL 1.1.x, parts of the patch should be merged. As of commit 9f9a199, the logic in ECChannel::HrSetCtx(ECConfig *lpConfig) is currently broken (always returns MAPI_E_CALL_FAILED) in the kc-8.7.x tag, always frees the freshly generated context, and on duplicate HrSetCtx calls leads to a memory leak of an SSL_CTX. Same thing goes for HrFreeCtx, which seems not to have been adapted for the use of std::atomic to refer to the SSL_CTX.

TomSchmidt

Unfortunately the above mentioned solution does not work for me. I still get ‘Error loading SSL context, POP3S and IMAPS will be disabled’.

System: debian 10, kopano 9.0.2

I run as root and did ‘setcap cap_net_bind_service=+ep /usr/sbin/kopano-gateway’

Do I have to completely restart kopano?

Regards, Tom

modelnine

@TomSchmidt the problem is one in the actual SSL_CTX setup of Kopano, which is currently broken (due to an incomplete and functionally broken patch which was applied just before christmas). The code seems to be similar in the 8.7.x and 9.0.x branches, so the first of the patches that I posted as PR 21 against the github branches (see https://github.com/Kopano-dev/kopano-core/pull/21/commits/60299e18d120d94f2c58ec75a354c73105015921) should apply to 9.0.x, too, to fix the invalid flow of control setting up the SSL context.

Please be aware that the patch restores the SSL-functionality of Kopano, but is incomplete in the sense that it is not a proper implementation of atomic accesses to SSL_CTX, which would require locking; basically, tacking on std::atomic for accesses to the SSL_CTX is not enough to actually make sure that reloading the SSL context while a Kopano server is operating does not lead to spurious segmentation faults due to race conditions (SSL_CTX can be used after being freed). So, don’t reload kopano-gateway to get new certificate material, always restart it, even with this patch.

If there’s interest, I’ll gladly supply a patch to implement the proper locking, but I gather that Kopano is working on something here and will fix this themselves at some point in time.

TomSchmidt

@modelnine,
thank you yery much to make things clear. This is quite sad while imaps isn’t state of the art but many still use it.
I’m wondering that there’s a major release upgrade and this isn’t working.

Stupid me tested in a sandbox but I did not test the imaps function. I cannot go back to 8.7

So I will wait for things coming.

modelnine

@fbartels is there any possibility of getting one of the two patchsets (or both) which are now in my Github-tree into Kopano some time soon? ;-) Basically, as it stands, for 9.0 and 8.7, (at least) all server-side SSL is currently broken. Thanks!

fbartels

Hi @modelnine,

as far as I know they are currently in review, but apart from my answers here I am not involved in this topic.

embexx

@TomSchmidt: I use stunnel to have imaps as long as kopano-gateway fails to provide imaps.

TomSchmidt

@embexx
Hi, nice idea, but I have to connect two Thunderbird-Users and TB can’t use z-push as AcitveSync. The addons promising ActiveSync are crap. One of them is my Mom using Win10. So this idea crashes here :-)

But thank a lot for replying!

tom

embexx

@TomSchmidt stunnel is like a proxy or SSL-wrapper. With stunnel iprovide imaps with kopano-gateway to connect my TB-users.
ActiveSync has not too much to do with that.

TomSchmidt

@embexx
OK, I got it, was on a wrong journey. I did the same and it works perfectly.

Thanks!

umgfoin

Hello @modelnine,
it looks like recent refactorings to HrEnableTLS, HrSetCtx and related are causing issues:
I’m getting SIGSEVs on SSL_accept (using openSSL 1.0.1e) , similar problems are discussed here.

Reverting commits introduced by PR22 fixes the issues on my unsupported system (CentOS6).

2020-01-30T12:28:10.497752: [kopano-gateway|T14538] [=======] Starting kopano-gateway version 10.0.1.89 (pid 14538 uid 482)
2020-01-30T12:28:10.498129: [kopano-gateway|T14538] [info   ] Re-using fd 5 for 0.0.0.0:143
2020-01-30T12:28:10.498157: [kopano-gateway|T14538] [info   ] Re-using fd 6 for [::]:143
2020-01-30T12:28:10.498248: [kopano-gateway|T14538] [info   ] Re-using fd 7 for 0.0.0.0:993
2020-01-30T12:28:10.498286: [kopano-gateway|T14538] [info   ] Re-using fd 8 for [::]:993
2020-01-30T12:28:10.500566: [kopano-gateway|T14538] [info   ] ECChannel::HrSetCtx(): SSL_CTX_NEW: Success.
2020-01-30T12:30:05.887132: [kopano-gateway|T14538] [info   ] Accepted connection from [2xx1:c22:xxxx:xxxx:1d52:xxxx:a949:xxxx]:56320
2020-01-30T12:30:05.887176: [kopano-gateway|T14538] [notice ] Starting worker thread for IMAPs request
2020-01-30T12:30:05.887360: [kopano-gateway|T14818] [warning] HTML safety filter is enabled in configuration, but KC is not compiled with libtidy
2020-01-30T12:30:05.887437: [kopano-gateway|T14818] [info   ] ECChannel::HrEnableTLS(): TLS flags 0x814a0bf7
2020-01-30T12:30:05.887474: [kopano-gateway|T14818] [crit   ] ----------------------------------------------------------------------
2020-01-30T12:30:05.887482: [kopano-gateway|T14818] [crit   ] Fatal error detected. Please report all following information.
2020-01-30T12:30:05.887490: [kopano-gateway|T14818] [crit   ] kopano-dagent 10.0.1.89
2020-01-30T12:30:05.887503: [kopano-gateway|T14818] [crit   ] OS: CentOS release 6.10 (Final)
 (Linux 3.10.0-957.12.2.vz7.86.2 x86_64)
2020-01-30T12:30:05.887515: [kopano-gateway|T14818] [crit   ] Thread name: kopano-gateway
2020-01-30T12:30:05.887528: [kopano-gateway|T14818] [crit   ] Peak RSS: 10024
2020-01-30T12:30:05.887536: [kopano-gateway|T14818] [crit   ] Pid 14538 caught SIGSEGV (11), traceback:
2020-01-30T12:30:05.887542: [kopano-gateway|T14818] [crit   ] Backtrace:
2020-01-30T12:30:05.887822: [kopano-gateway|T14818] [crit   ] f0. /usr/lib64/libkcutil.so.0(+0x51c2b) [0x7f21332c5c2b]
2020-01-30T12:30:05.887835: [kopano-gateway|T14818] [crit   ] f1. /usr/lib64/libkcutil.so.0(+0x37f7f) [0x7f21332abf7f]
2020-01-30T12:30:05.887843: [kopano-gateway|T14818] [crit   ] f2. /usr/lib64/libkcutil.so.0(+0x3a003) [0x7f21332ae003]
2020-01-30T12:30:05.887850: [kopano-gateway|T14818] [crit   ] f3. /lib64/libpthread.so.0() [0x329f00f7e0]
2020-01-30T12:30:05.887858: [kopano-gateway|T14818] [crit   ] f4. /usr/lib64/libssl.so.10(SSL_accept+0x1) [0x3b2cc42c41]
2020-01-30T12:30:05.887865: [kopano-gateway|T14818] [crit   ] f5. /usr/lib64/libkcutil.so.0(_ZN2KC9ECChannel11HrEnableTLSEv+0xce) [0x7f213329dbae]
2020-01-30T12:30:05.887873: [kopano-gateway|T14818] [crit   ] f6. /usr/sbin/kopano-gateway() [0x4102c8]
2020-01-30T12:30:05.887880: [kopano-gateway|T14818] [crit   ] f7. /lib64/libpthread.so.0() [0x329f007aa1]
2020-01-30T12:30:05.887887: [kopano-gateway|T14818] [crit   ] f8. /lib64/libc.so.6(clone+0x6d) [0x329e8e8c4d]
2020-01-30T12:30:05.887901: [kopano-gateway|T14818] [crit   ] Signal errno: Success, signal code: 1
2020-01-30T12:30:05.887909: [kopano-gateway|T14818] [crit   ] Sender pid: 48, sender uid: 0, si_status: 0
2020-01-30T12:30:05.887916: [kopano-gateway|T14818] [crit   ] Signal value: 0, faulting address: 0x30
2020-01-30T12:30:05.887931: [kopano-gateway|T14818] [crit   ] When reporting this traceback, please include Linux distribution name (and version), system architecture and Kopano version.

Best regards,
umgfoin.

koole

Hi guys,

What is the status against kopano-gateway in Debian 10?

We have experienced also same problem (many times in a hour; autorestart works partly):

Tue Feb 25 14:19:07 2020: [ZGateway IMAPs|T542] [crit ] #20. /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f243aa8b4cf]
Tue Feb 25 14:19:07 2020: [ZGateway IMAPs|T542] [crit ] Signal errno: Success, signal code: 1

in kopano-gateway 8.7.0-3 (comes from Debian buster main section)

What should I do - downgrade system to Debian stretch/9 and use some supported kopano-core (and kopano-gateway) version or is there somewhere working kopano-core/gateway system against Debian 10 already?

All we know that Debian 9 is already old-stable. So better solution might be Ubuntu 18.04 then?

(sorry - maybe there is a better thread…)

/Mikko

jengelh

The status is that all is in good order in the source codes as of Feb 1 2020, corresponding to v9.0.3 and v10.0.1.