Can't contact LDAP server
-
Hi @darootler,
we are not publishing release notes/change logs for development versions, only for proper releases.
But you could check https://stash.kopano.io/projects/KC/repos/kopanocore/browse/RELNOTES.txt and https://stash.kopano.io/projects/KC/repos/kopanocore/commits.
-
Samba logs:
[2020/08/04 14:28:03.041662, 3] ../source4/smbd/service_stream.c:65(stream_terminate_connection) Terminating connection - 'ldapsrv_call_loop: tstream_read_pdu_blob_recv() - NT_STATUS_IO_TIMEOUT' [2020/08/04 14:28:03.041856, 2] ../source4/smbd/process_standard.c:473(standard_terminate) standard_terminate: reason[ldapsrv_call_loop: tstream_read_pdu_blob_recv() - NT_STATUS_IO_TIMEOUT] [2020/08/04 14:28:03.046319, 2] ../source4/smbd/process_standard.c:157(standard_child_pipe_handler) Child 6044 () exited with status 0 [2020/08/04 14:28:03.074973, 3] ../source4/smbd/service_stream.c:65(stream_terminate_connection) [2020/08/04 14:28:03.074973, 3] ../source4/smbd/service_stream.c:65(stream_terminate_connection) Terminating connection - 'ldapsrv_call_loop: tstream_read_pdu_blob_recv() - NT_STATUS_IO_TIMEOUT' Terminating connection - 'ldapsrv_call_loop: tstream_read_pdu_blob_recv() - NT_STATUS_IO_TIMEOUT' [2020/08/04 14:28:03.075211, 2] ../source4/smbd/process_standard.c:473(standard_terminate) standard_terminate: reason[ldapsrv_call_loop: tstream_read_pdu_blob_recv() - NT_STATUS_IO_TIMEOUT] [2020/08/04 14:28:03.075257, 2] ../source4/smbd/process_standard.c:473(standard_terminate) standard_terminate: reason[ldapsrv_call_loop: tstream_read_pdu_blob_recv() - NT_STATUS_IO_TIMEOUT]
Kopano logs
Aug 4 14:28:04 hostname kopano-server[1301]: LDAP search error: Can't contact LDAP server. Will unbind, reconnect and retry. Aug 4 14:28:05 hostname kopano-server[1301]: Authentication by plugin failed for user "USERNAME": LDAP auth for user "USERNAME": Can't contact LDAP server
So for me it seems that the ldap server is closing the connection due to timeout and then kopano-server is reconnecting to ldap. If during this time an authentication is happening i am getting these errors. I was nat facing this issue in version 13, so there must a be change on kopano side.
Any ideas? I cannot believe that i am the only one facing this issue.
Regards
Richard -
Maybe https://stash.kopano.io/projects/KC/repos/kopanocore/commits/098aa87298fe9212fe4c56d538097d3bb435aba5 is the cause of my issue…
Here is relevant the output from “kopano-stats --system”:
ldap_connect Number of connections made to LDAP server 39 ldap_reconnect Number of re-connections made to LDAP server 10 ldap_connect_fail Number of failed connections made to LDAP server 0 ldap_connect_time Total duration (µs) of connections made to LDAP server 1484275 ldap_max_connect Longest connection time (µs) made to LDAP server 70662 ldap_auth Number of LDAP authentications 3179 ldap_auth_fail Number of failed authentications 15 ldap_auth_time Total authentication time (µs) 56987108 ldap_max_auth Longest duration (µs) of authentication made to LDAP server 67767 ldap_avg_auth Average duration (µs) of authentication made to LDAP server 18433 ldap_search Number of searches made to LDAP server 8002 ldap_search_fail Number of failed searches made to LDAP server 0 ldap_search_time Total duration (µs) of LDAP searches 8692105 ldap_max_search Longest duration (µs) of LDAP search 49338 userplugin User backend plugin ldap
ldap_auth_fail reveals my problem i think.
Regards
Richard -
@darootler said in Can't contact LDAP server:
So for me it seems that the ldap server is closing the connection due to timeout and then kopano-server is reconnecting to ldap. If during this time an authentication is happening i am getting these errors.
That should not really matter, since afair for the login a bind is issued and this happens on a new connection anyways. Ah that seems to have changed with the linked commit.
-
Well, i just reviewed the logs again and every time the ldap server resets the connection and an authentication is made i am getting this error. False username/password isn’t the case because the credentials are stored and not entered manually. I am sure that has something to do with the changes mentioned above, could you be so kind and contact Jan Engelhardt for a quick review of my findings?
Otherwise i have to downgrade kopano, and that’s hard like hell with all the dependencies.
Regards
Richard -
Even more debug information:
Kopano logs:
Aug 4 17:37:29 serverv2 kopano-server[1330]: Authentication by plugin failed for user "USERNAME": LDAP auth for user "USERNAME": Can't contact LDAP server Aug 4 17:37:31 serverv2 kopano-server[1330]: Authentication by plugin failed for user "USERNAME": LDAP auth for user "USERNAME": Can't contact LDAP server
tcpdump from kopano lookup:
17:37:29.128581 IP kopano-server.49129 > ldap-server.domain: 61640+ AAAA? kopano-server. (42) 17:37:29.128806 IP ldap-server.shell > kopano-server.46178: Flags [.], ack 2058, win 9434, options [nop,nop,TS val 3865952505 ecr 2989291037], length 0 17:37:29.129298 IP ldap-server.domain > kopano-server.49129: 61640* 0/1/1 (177) 17:37:29.129652 IP kopano-server.57704 > ldap-server.domain: 47351+ AAAA? kopano-server. (26) 17:37:29.130077 IP ldap-server.domain > kopano-server.57704: 47351 0/0/1 (37) 17:37:29.130528 IP kopano-server.56223 > ldap-server.domain: 54523+ AAAA? kopano-server. (42) 17:37:29.130994 IP ldap-server.domain > kopano-server.56223: 54523* 0/1/1 (177) 17:37:29.131039 IP kopano-server.39715 > ldap-server.domain: 34035+ AAAA? kopano-server.my.domain. (58) 17:37:29.131504 IP ldap-server.domain > kopano-server.39715: 34035 NXDomain* 0/0/0 (58)
tcpdump from shell nslookup:
17:41:46.072987 IP kopano-server.57054 > ldap-server.domain: 53018+ A? kopano-server. (42) 17:41:46.073725 IP ldap-server.domain > kopano-server.57054: 53018* 1/1/1 A 192.168.2.3 (217) 17:41:46.073877 IP kopano-server.55171 > ldap-server.domain: 25675+ AAAA? kopano-server. (42) 17:41:46.074676 IP ldap-server.domain > kopano-server.55171: 25675* 0/1/1 (177)
My DNS server cannot answer the AAAA request because i am not using IPv6. On the kopano server i have disabled IPv6 on grub level. So why is kopano trying to resolve the name of the LDAP server via IPv6 lookups?
Maybe that’s the cause of this issue.
Regards
Richard -
So why is kopano trying to resolve the name of the LDAP server via IPv6 lookups?
First, that’s glibc, second, because the content of data transmissions and castrating a system’s ability to offer AF_INET6 sockets are two separate things. It helps to know what ipv6.disable does, and more importantly, doesn’t do.
-
@jengelh said in Can't contact LDAP server:
So why is kopano trying to resolve the name of the LDAP server via IPv6 lookups?
First, that’s glibc, second, because the content of data transmissions and castrating a system’s ability to offer AF_INET6 sockets are two separate things. It helps to know what ipv6.disable does, and more importantly, doesn’t do.
Thank you for clarification, i think i fixed all my DNS issues playing around with systemd-resolve but i am still facing issues with kopano-server authentication issues after upgrading to 10.0.6.356:
ldap_connect Number of connections made to LDAP server 27 ldap_reconnect Number of re-connections made to LDAP server 0 ldap_connect_fail Number of failed connections made to LDAP server 0 ldap_connect_time Total duration (µs) of connections made to LDAP server 953515 ldap_max_connect Longest connection time (µs) made to LDAP server 54989 ldap_auth Number of LDAP authentications 129 ldap_auth_fail Number of failed authentications 10 ldap_auth_time Total authentication time (µs) 3188436 ldap_max_auth Longest duration (µs) of authentication made to LDAP server 58361 ldap_avg_auth Average duration (µs) of authentication made to LDAP server 24737 ldap_search Number of searches made to LDAP server 457 ldap_search_fail Number of failed searches made to LDAP server 0 ldap_search_time Total duration (µs) of LDAP searches 426920 ldap_max_search Longest duration (µs) of LDAP search 2977 userplugin User backend plugin ldap
Any hints?
Regards
Richard -
A pull request is under review.
-
After investing about 2 days i just downgraded to version 10.0.6.13.
Regards
Richard -
Any updates here?
Regards
Richard -
included in >= 10.0.6.428.
-
I am on Kopano Core Version core-10.0.6.406.e263d46-Debian_10-amd64 and experience the same ldap issue since the upgrade with the ldap service running on Univention UCS.
osrelease Pretty operating system name Debian GNU/Linux 10 (buster) program_name Program name kopano-server program_version Program version 10.0.6 server_guid 7636E72C7285493384B2CCC3487C6508
Interesting enough, under https://download.kopano.io/community/core%3A/ I find only the 10.0.6.406 Version since 13.08.2020. I thought these artefacts were built daily? Anyway, when do you expect version 428+ with the ldap fix to be available?
Should you require more information to debug the issue, I’m happy to assist.
-
@jengelh said in Can't contact LDAP server:
included in >= 10.0.6.428.
Does anyone know when this, or a later, version will be available on https://download.kopano.io/community/core%3A/ ? There has been no update on the nightly builds since 13.08.2020. Or can anyone point me to the correct repo to compile it myself?
-
@alexthetiger68 said in Can't contact LDAP server:
Or can anyone point me to the correct repo to compile it myself?
The sources for the Groupware Core part are located at https://stash.kopano.io/projects/KC/repos/kopanocore/browse
The last I checked the missing nightly releases were looked into, but currently the focus is on finalizing the upcoming release for customers, so the nightly downloads are not the highest priority.
-
Good news, everyone.
I just upgraded to core-10.0.6.496 and so far, the ldap connection errors seems to be gone.
-
Well, the error messages have changed slightly, but at least the clients (Outlook via Activesync w/o KOE, Mail on iOS) do not seem to complain anymore.
Previously the clients would show a popup stating that the username/password was wrong, which I just had to <cancel> multiple times till it went away and synching of Kopano server to the clients resumed.
The new error message:
2020-10-08T06:39:27.011248: [error ] Previous message logged 76 times
2020-10-08T06:39:27.011824: [error ] K-1585: LDAP auth error: Can’t contact LDAP server. Will rebind & retry.
2020-10-08T06:39:27.065428: [error ] K-1582: LDAP search error: Can’t contact LDAP server. Will reconnect and retry.