Authentication Between the various components in a distributed setup


  • So I’m trying to set up a configuration such that the different components are spread out and communicate over the network rather than unix sockets. That has however hit a bit of a snag rather early on. I’ve set up a mysql cluster, as well as an ldap cluster. I’ve also then configured a kopano-server component. Since it wants to connect outwards to search, I figured I’d start by setting up that as the next component. I have my own CA, that has generated certs for both server and search. server.cfg has server_ssl_ca_file set to the ca.crt, it has server_ssl_key_file set to the pem with both crt and key for the server. It also has sslkeys_path set to a path, that contains the search.crt file. search.cfg in turn is configured with sslkey_file to be its file for both crt and key, and ssl_private_key_file to the search.key and ssl_certificate_file to the search.crt. Starting search now however does not allow the two to communicate. Search tries to log on to server as a SYSTEM user and fails. But I can find no way to configure a user to log in as or users to allow. I only find the allow_local_users option but that’s about which users are allowed in using the unix socket, which I’m not using.

    From Search the log is:
    [error ] HrLogon server “https://kopano-server:237” user “SYSTEM”: logon failed
    2020-04-16 12:40:19,010 - search - ERROR - Traceback (most recent call last):
    File “/usr/lib/python3/dist-packages/kopano/server.py”, line 320, in init
    flags=flags)
    File “/usr/lib/python3/dist-packages/MAPI/Util/init.py”, line 72, in OpenECSession
    admin.ConfigureMsgService(uid, 0, 0, profprops)
    File “/usr/lib/python3/dist-packages/MAPICore.py”, line 1661, in ConfigureMsgService
    return _MAPICore.IMsgServiceAdmin_ConfigureMsgService(self, lpUID, ulUIParam, ulFlags, cValues)
    MAPI.Struct.MAPIErrorLogonFailed: MAPI error 80040111 (MAPI_E_LOGON_FAILED)

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File “/usr/lib/python3/dist-packages/kopano/service.py”, line 182, in start
    config=self.config, service=self)
    File “/usr/lib/python3/dist-packages/kopano/service.py”, line 101, in _daemonize
    _daemon_helper(func, service, log)
    File “/usr/lib/python3/dist-packages/kopano/service.py”, line 46, in _daemon_helper
    func()
    File “/usr/lib/python3/dist-packages/kopano_search/init.py”, line 304, in main
    self.state_db = os.path.join(index_path, self.server.guid+’_state’)
    File “/usr/lib/python3/dist-packages/kopano/service.py”, line 167, in server
    _skip_check=True)
    File “/usr/lib/python3/dist-packages/kopano/server.py”, line 332, in init
    password incorrect’)
    kopano.errors.LogonError: Could not logon to server: username or password incorrect

    From server, the log is:
    2020-04-16T17:18:33.732995: [warning] K-2171: soap_ssl_accept: SSL_ERROR_SYSCALL
    Error observed by underlying SSL/TLS BIO: Success (SSL_accept() failed in soap_ssl_accept())

    Based on the errors, I would guess that it’s not actually a username/password issue but rather that something is going wrong in the tls handshaking, but I don’t understand what :/ The certs are all using rsa encryption which seems common enough that it should work, though I can’t actually find anything that says which are and not.

  • Kopano

    Hi @EtherMan,

    the base for what you want to achieve is explained in https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#multi-server-setup. There is no user to configure when going through tcp, but authentification is done through ssl client certs (in the end afair it’s represented as the system user again).

    PS: generally speaking its always helpful when the used versions are noted. If you are looking into this for a commercial deployment then I would also recommend to get in touch with our support.


  • No that document does not at all describe the situation I’m trying to achieve. It’s still a single kopano-server, they’re just communicating over TCP rather than unix sockets. I’m not talking about distributed kopano with user stores spread out over multiple kopano-servers, it’s a single kopano-server, it’s just running in a different container than in this case, the search container. As for auth going through the certs is what I figured, except how? It’s seemingly not enough that search’s cert is in and the CA being the same. There’s also missing info on compatible certs, such as what cryptos are supported.

    As for version, 10.0.3.41 and no, it’s not for commercial deployment. To my mind, Kopano isn’t up to that task since it doesn’t support actual HA, only failover, which just isn’t good enough. Especially when it lacks both fencing and quorum. But that’s not really relevant to this issue.

  • Kopano

    @EtherMan multiple kopano-server instances would just be the next step, but having components such as dagent communicate with the server through tcp is required here as well, see https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#creating-ssl-certificates

    And of course can you achieve ha with Kopano, you just need an outside “orchestrator” like for example pacemaker.


  • @EtherMan said in Authentication Between the various components in a distributed setup:

    As for auth going through the certs is what I figured, except how? It’s seemingly not enough that search’s cert is in and the CA being the same. There’s also missing info on compatible certs, such as what cryptos are supported.

    Whatever your TLS library offers. KC does not care.

    Cert guide: https://stash.kopano.io/projects/KC/repos/kopanocore/browse/doc/adminssl.txt . (Each paragraph stands on its own; if you don’t want subjectAltNames, you don’t need it.)


  • Problem with that document @fbartels is that it presupposes the use of a script, that may not at all be what is being used. Such as in k8s, you’d normally use cert-manager and not at all that script. While it’s possible to read that script to get some sample commands, that still doesn’t actually say what it is it’s actually needing.

    And no, you can’t get HA with as an example pacemaker in kopano. Not the way the docs claim it at least. A baseline requirement for HA is multiple live nodes that sync together. If a node goes down, it’s not enough that pacemaker now goes in an relaunches on another node which the docs do it. That’s a failover, NOT High Availability. High Availability as a term requires that at worst, only the connections currently active on the failed node is dropped. Any new connections should be redirected to live nodes. Or even better where systems allow for gracefully just having the existing connection just moved to a different node and just have a small lagspike. Having as Kopano does it with having pacemaker start up the service on a different node if it fails… Yea that’s failover, not HA. Just because you can have a HA setup with pacemaker, does not make every setup that happens to use it a HA setup. If you think of it like this. In a large setup, it may actually take upwards an hour or two to actually start a service after even a basic reboot. With kopano’s failover… That means the service would be down for that entire hour. If both nodes are active, with one just pulling sync data from the other… Well then it’s just a matter of detecting when the primary has failed. And that only takes as long as the heartbeats which if you want, you could set to like 10 per second if you wish. Now all that could in theory be solved by having multiple kopano-server instances that are all the same server as far as kopano knows, as in all synced to the same database backend that in itself is HA and so on. But the docs specifically say that is not a valid setup.

    @jengelh Ok so it doesn’t care what crypto it is, thank you, but that “guide” doesn’t answer what I asked either. I’m not asking how to generate a certificate or key. I’m well aware of how to do so. I’m asking what format kopano expects it to be in, since there seems to be special requirements as to that. PEM? DER? PKCS12? PKCS7? What uses does the certificates need to have? Subject? Commonnames? DNSes? IPs? uriSANs? emailSANs? min/max durations, is a shared CA required? Or perhaps forbidden? and so on and so on… As well as which of the parts of this needs to go where exactly.


  • EtherMan,

    a few weeks ago I did a similar setup, to answer your questions:

    Certificates as PEM files are needed, the Subject ‘-subj “/C=XX/ST=State/O=Customer/CN=kopano-mail.customer.xx”’ is the LDAP server name in certificate.


  • @WalterHof Thank you. That’s one part of it solved. I think I already had them as PEM but perhaps something wasn’t. Reissued them and now it works. Now I just need to figure out how to get cert manager to actually manage combined privkey+cert, as well as extracting the pubkey from the cert. Though that’s not for here :)


  • @EtherMan said in Authentication Between the various components in a distributed setup:

    to actually manage combined privkey+cert

    Simply copy both file into one file.
    cat key.pem cert.pem > combined.pem


  • @WalterHof That’s not really how Cert Manager works. It’s a bit more complicated than that. Basically you define how you want the cert to look, and cert manager issues you a cert, and private key, and inserts them as secrets in k8s. So to do that, I have to first extract the secret, decode the secret, combine, encode, and insert it back in. It keeps track of pretty much everything for you in that regard. But because kopano needs extracted pubkeys and combined files. Well those are manual so will not get renewed together with the certificate as managed by Cert Manager.


  • @EtherMan said in Authentication Between the various components in a distributed setup:

    Ok so it doesn’t care what crypto it is, thank you, but that “guide” doesn’t answer what I asked either. I’m not asking how to generate a certificate or key. I’m well aware of how to do so. I’m asking what format kopano expects it to be in, since there seems to be special requirements as to that. PEM? DER? PKCS12? PKCS7?

    The commands in adminssl.txt produce PEM files. adminssl.txt also says the files should contain “PRIVATE KEY” / “CERTIFICATE” markers, which is something you would normally find only in PEM.

    What uses does the certificates need to have?

    Again, KC mostly does not care. You really just have to please your system’s openSSL default config. There is just hostname (subject) validation, i.e. connecting to https://ab.cd:237/ requires the cert to have “ab.cd” in either CN or SAN.


  • As I said though, the command itself does nothing to answer the question of acceptable formats, and clearly it’s NOT enough that the certs are in a format accepted by openssl, since that accepts other formats as well. It’s also completely incorrect KC doesn’t care about uses, because I’ve been over this now and despite being client auth certs, Kopano requires the certs to be server auth as well, even when being used only for client auth. And no, it doesn’t do subject validation either. It may do it for the server, I have not actually tested that, but this is about the client certs. As in the cert the client is presenting to the server. It’s not connecting fom an URL to have a subject line like that. Anyway, as I’ve said, this part is finished. Search is working. Trying to get konnect and webapp to work is a whole new level of stuff to deal with. It’s in times like these where you try to do anything even just slightly out of the box, that you find out how poorly documented a project like this truly is.