kopano-spamd causes endless cpu-load

mhope

Dear Kopano-Team,
when moving email to the junkfolder, kopano-spamd first works as expected and inotify-spamlearn adds the mail to the spam database and afterwards deletes the eml file.
So far - so good. Then the kopano-spamd-process causes 100% cpu-load which can only be stopped by restarting the kopano-spamd with systemctl restart kopano-spamd.

When stopping kopano-spamd the logs say:

un 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]: 2020-06-17 21:17:02,723 - spamd - INFO - stopping spamd
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]: Terminating on signal 15
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]: Exception ignored in: <bound method _ConnectionBase.__del__ of <multiprocessing.connection.Connection object at 0x7f7f393dd080>>
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]: Traceback (most recent call last):
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]:   File "/usr/lib/python3.6/multiprocessing/connection.py", line 132, in __del__
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]:     self._close()
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]:   File "/usr/lib/python3.6/multiprocessing/connection.py", line 361, in _close
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]:     _close(self._handle)
Jun 17 21:17:02 kopanoubuntuamabeja kopano-spamd[2320]: OSError: [Errno 9] Bad file descriptor

Environment is:

Ubuntu 18.04.02 Server (VM)
Kopano Version 10.0.5.115
MariaDB 10.1.44
kopano-spamd running with spamassassin

spamd.cfg:

##############################################################
# SPAMD SERVICE SETTINGS

# run as specific user
run_as_user         = kopano

# run as specific group
run_as_group        = amavis

# control pid file
pid_file            =   /var/run/kopano/spamd.pid

# run server in this path (when not using the -F switch)
# running_path = /var/lib/kopano

##############################################################
# LOG SETTINGS

# Logging method (syslog, file)
log_method          =   syslog

# Loglevel (0(none), 1(crit), 2(err), 3(warn), 4(notice), 5(info), 6(debug))
log_level           =   6

# Logfile for log_method = file, use '-' for stderr
log_file            =   /var/log/kopano/spamd.log

# Log timestamp - prefix each log line with timestamp in 'file' logging mode
log_timestamp       =   1

###############################################################
# SPAMD Specific settings

# The dir where spam mails are written to which are later picked up
# by the sa-learn program
spam_dir = /var/lib/kopano/spamd/spam

# Location for the database containing metadata on learned spam
spam_db = /var/lib/kopano/spamd/spam.db

# Learn ham, when the user moves emails from junk to inbox,
# enabled by default.
earn_ham = yes

# The dir where ham mails are written to which are later picked up
# by the sa-learn program
ham_dir = /var/lib/kopano/spamd/ham

# Spamassassin group
sa_group = amavis

# Header tag for spam emails
header_tag = X-Spam-Flag

Anyone else encountering this issue? Any idea why this error occurs?

Kind regards and I’d again like to thank the whole Kopano-Team for developing and maintaining such a masterpiece of software!

Marco

mhope

Dear Kopanoers :),

unfortunately a complete reinstall of Kopano did not solve the issue.

After installing the basics I optimized everything, added amavis and spamassassin, adapted the postfix-config-files and integrated the inotify-spamlearn as well. In order to make inotify-spamlearn work I needed to install a few python3-modules as well and I updated Python3.
Finally I added pyzor and ryzor to the spamassassin-database located in /var/lib/amavis/.spamassassin

When receiving mail, the logs clearly show that the message is transfered and checked by spamassassin - spam messages are filtered perfectly, are marked as spam and transfered into the junk-folder.

When manually moving a message into the junk-folder kopano-spamd recognizes this, copies the message into the configured spam-learning-folder. Inotify-Spamlearn picks it up, trains the spamassassin database and everything seems fine.

However - further observing processes via top shows, that kopano-spamd does not return to idle-state but instead is draining cpu-time.

I am still wondering why I seem to be the only one with this problem. Does anybody have an idea what might cause this issue?

Here are the logs corresponding to my observations:

Kopano-Spamd-process before moving spam to junk-folder
Bildschirmfoto zu 2020-07-10 18-37-43.png

Kopano-Spamd-process after moving spam to junk-folder
Bildschirmfoto zu 2020-07-10 18-40-14.png

Systemlog shows that spamd works as expected and that inotify-spamlearn is processing the message and learning the token:

Jul 10 18:39:39 KopanoLXC kopano-spamd[1373]: 2020-07-10 18:39:39,713 - spamd - INFO - Learning message as SPAM, entryid: 000000007876DD3EBCC94E81B4C119C2E064F76A01000000050000005E9820A
Jul 10 18:39:41 KopanoLXC inotify-spamlearn.py[1140]: config: path "/dev/null/.spamassassin" is inaccessible: Not a directory
Jul 10 18:39:41 KopanoLXC inotify-spamlearn.py[1140]: config: path "/dev/null/.spamassassin/user_prefs" is inaccessible: Not a directory
Jul 10 18:39:42 KopanoLXC inotify-spamlearn.py[1140]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/5E5CCD41152A4F06809A702D64DC3675.eml: Learned tokens from 1 message(s) (1 mess
Jul 10 18:39:42 KopanoLXC inotify-spamlearn.py[1140]: INFO Removing file: /var/lib/kopano/spamd/spam/5E5CCD41152A4F06809A702D64DC3675.eml

The config-path causing the inotify-spamlearn-message can safely be ignored since the path of the config-file is manually given to sa-learn. This warning is probably due to the fact that Kopano is running inside an Ubuntu 18.04-LXC Container. The errors have been exactly the same when running inside aan Ubuntu 18.04 -VM.

When stopping the kopano-spamd-service the logs say:

Jul 10 18:45:51 KopanoLXC systemd[1]: Stopping Kopano Groupware Core SPAM learning Daemon...
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]: Terminating on signal 15
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]: Exception ignored in: <bound method _ConnectionBase.__del__ of <multiprocessing.connection.Connection object at 0x7fe4d67e9c50>>
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]: Traceback (most recent call last):
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]:   File "/usr/lib/python3.6/multiprocessing/connection.py", line 132, in __del__
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]:     self._close()
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]:   File "/usr/lib/python3.6/multiprocessing/connection.py", line 361, in _close
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]:     _close(self._handle)
Jul 10 18:45:51 KopanoLXC kopano-spamd[1373]: OSError: [Errno 9] Bad file descriptor
Jul 10 18:45:51 KopanoLXC systemd[1]: kopano-spamd.service: Main process exited, code=exited, status=1/FAILURE
Jul 10 18:45:51 KopanoLXC systemd[1]: kopano-spamd.service: Failed with result 'exit-code'.
Jul 10 18:45:51 KopanoLXC systemd[1]: Stopped Kopano Groupware Core SPAM learning Daemon.

Any ideas what I might search for in order to solve this?

Kind regards,

Marco

mhope

Dear Kopanoers,

finally I could track down the problem. It was due to conflicting python3-versions.

I switched to an older snapshot of my fresh installation and continued everything without manually updating the python-libraries and everything works fine.

Sorry that it took so long for me to find out.

Kind regards,

Marco

PS: Is there a way to mark this als “solved”? I will alter the subject if possible.