Navigation

    Kopano
    • Register
    • Login
    • Search
    • Categories
    • Get Official Kopano Support
    • Recent
    Statement regarding the closure of the Kopano community forum and the end of the community edition

    Define ham and spam for spamassassin

    Kopano Groupware Core
    5
    29
    4158
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ashceryth
      ashceryth last edited by ashceryth

      Hi @BMWfan

      I think this is because the default home of the kopano user is /dev/null (you can check your /etc/passwd) and inotify-spamlearn is running as kopano user. It seems like ~/.spamassassin/ (where ~/ is the home directory of the kopano user - /dev/null) is the default directory used by the sa-learn command. You can specify the paths used by the sa-learn command in inotify-spamlearn.cfg or run inotify-spamlearn as different user (set in the systemd unit file).

      Regards
      Patrick

      BMWfan 1 Reply Last reply Reply Quote 0
      • BMWfan
        BMWfan @ashceryth last edited by BMWfan

        @ashceryth thanks for your response. I’am thinking about it what would be the best choice todo.
        I’am not sure if it would help and would be good if i assign the kopano user a home directory, because /dev/null is standard if you dont set once i think. What have you done to get it work?

        For kopano-spamd i assigned already kopano to the amavis group

        gpasswd -a kopano amavis
        
        1 Reply Last reply Reply Quote 0
        • ashceryth
          ashceryth last edited by ashceryth

          I suggest to try to run inotify-spamlearn as the same user as amavis runs as.

          So, edit the systemd unit file (/etc/systemd/system/inotify-spamlearn.service) and set:

          User=amavis
          Group=amavis
          

          Run systemctl daemon-reload after editing and restart the service with systemctl restart inotify-spamlearn.

          Check your spamd.cfg to make sure it uses the amavis group (the .eml files should then get the gid of the amavis group):

          sa_group = amavis

          Does it work then?

          I don’t use the sa-learn command. I use Rspamd and let inotify-spamlearn send the mails via HTTP post requests with curl.

          BMWfan 1 Reply Last reply Reply Quote 0
          • BMWfan
            BMWfan @ashceryth last edited by BMWfan

            @ashceryth said in Define ham and spam for spamassassin:

            systemctl restart inotify-spamlearn

            No, this solves it not. The problem is after i change the sa_group at /etc/kopano/spamd.cfg to amavis then i get permission errors in the kopano-spamd process as here described: Kopano spamd error - PermissionError: [Errno 1] Operation not permitted

            My configuration is now this:
            /etc/kopano/spamd.cfg

            # run as specific user
            run_as_user         = kopano
            
            # run as specific group
            run_as_group        = kopano
            
            # Spamassassin group
            sa_group = kopano
            

            /etc/systemd/system/inotify-spamlearn.service

            [Service]
            User=kopano
            Group=kopano
            

            but then again with this log entries:

            Jan 12 16:13:10 euve264608 inotify-spamlearn.py[7348]: config: path "/dev/null/.spamassassin" is inaccessible: Not a directory
            Jan 12 16:13:10 euve264608 inotify-spamlearn.py[7348]: config: path "/dev/null/.spamassassin/user_prefs" is inaccessible: Not a directory
            Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.euve264608.serverprofi24.de.7427 for /var/lib/spamassassin/bayes.lock: Permission denied
            Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.euve264608.serverprofi24.de.7427 for /var/lib/spamassassin/bayes.lock: Permission denied
            Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 500.
            Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/72C00ABE4488476D8AF626120D868211.eml: Learned tokens from 0 message(s) (1 message(s) examined)
            Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: INFO Removing file: /var/lib/kopano/spamd/spam/72C00ABE4488476D8AF626120D868211.eml
            

            if i change only these entries from kopano to amavis
            /etc/systemd/system/inotify-spamlearn.service

            [Service]
            User=amavis
            Group=amavis
            

            then i get this messages:

            Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Starting inotify-spamlearn.py
            Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Looking for existing files in /var/lib/kopano/spamd/spam
            Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Finished looking for existing files in /var/lib/kopano/spamd/spam
            Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Looking for existing files in /var/lib/kopano/spamd/ham
            Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Inotify learning started
            Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Finished looking for existing files in /var/lib/kopano/spamd/ham
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.Servername.serverprofi24.de.7506 for /var/lib/spamassassin/bayes.lock: Permission denied
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.Servername.serverprofi24.de.7506 for /var/lib/spamassassin/bayes.lock: Permission denied
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 500.
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml: Learned tokens from 0 message(s) (1 message(s) examined)
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: INFO Removing file: /var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: Exception in thread Inotify Handling:
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: Traceback (most recent call last):
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 80, in inotified
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     process('/'.join([watch_path, filename]), spamcmd, delete, initiator)
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 43, in process
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     os.remove(filename)
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: PermissionError: [Errno 13] Permission denied: '/var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml'
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: During handling of the above exception, another exception occurred:
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: Traceback (most recent call last):
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     self.run()
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/lib/python3.5/threading.py", line 862, in run
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     self._target(*self._args, **self._kwargs)
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 85, in inotified
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     i.remove_watch([spam_dir, ham_dir])
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/lib/python3.5/dist-packages/inotify/adapters.py", line 109, in remove_watch
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     wd = self.__watches.get(path)
            Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: TypeError: unhashable type: 'list'
            

            These are the amavis and kopano rights under /etc/groups

            amavis:x:123:clamav,kopano
            kopano:x:999:kapi,konnect,amavis
            
            ashceryth 1 Reply Last reply Reply Quote 0
            • ashceryth
              ashceryth last edited by ashceryth

              @BMWfan said in Define ham and spam for spamassassin:

              No, this solves it not. The problem is after i change the sa_group at /etc/kopano/spamd.cfg to amavis then i get permission errors in the kopano-spamd process as here described: Kopano spamd error - PermissionError: [Errno 1] Operation not permitted

              Maybe because the problem described by @fbartels in this post insn’t solved for you yet?

              Who is the owner/group and what are the permissions of the files in /var/lib/kopano/spamd/spam/ (just ls -la)?

              And please test the sa-learn command manually as the desired user before configuring it with inotify-spamlearn to see if everything works as expected.

              BMWfan 1 Reply Last reply Reply Quote 0
              • BMWfan
                BMWfan @ashceryth last edited by

                @ashceryth thanks for your reponse and hints.
                Here is the suggested output:

                $ /var/lib/kopano/spamd/spam# ls -la
                total 8
                drwxr-xr-x 2 kopano kopano 4096 Jan 12 17:05 .
                drwxr-xr-x 4 kopano kopano 4096 Dec 27 10:32 ..
                
                1 Reply Last reply Reply Quote 0
                • ashceryth
                  ashceryth @BMWfan last edited by

                  Well, at the moment your spam folder is empty. Normally the .eml files get deleted after learning by inotify-spamlearn. However, according to your log from above it wasn’t possible to remove them because of permission issues:

                  @BMWfan said in Define ham and spam for spamassassin:

                  Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 43, in process
                  Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     os.remove(filename)
                  Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: PermissionError: [Errno 13] Permission denied: '/var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml'
                  

                  That’s why I’m asking. Would be interesting to see the permissions of the .eml files themselves.

                  BMWfan 1 Reply Last reply Reply Quote 0
                  • BMWfan
                    BMWfan @ashceryth last edited by BMWfan

                    @ashceryth i stopped in the inotify-spamlearn service and here is the suggested output:

                    $ /var/lib/kopano/spamd/spam# ls -la
                    total 12
                    drwxr-xr-x 2 kopano kopano 4096 Jan 14 20:39 .
                    drwxr-xr-x 4 kopano kopano 4096 Dec 27 10:32 ..
                    -rw-rw-rw- 1 kopano kopano 3643 Jan 14 20:39 600F5F44181B4F2B865EAE2A45EC2573.eml
                    
                    1 Reply Last reply Reply Quote 0
                    • ashceryth
                      ashceryth last edited by

                      The reason for the permission error messages is that your spam directory is not writable for the group. Without write permissions the .eml files cannot be removed by inotify-spamlearn running as amavis (member of kopano group).

                      Just run chmod g+w /var/lib/kopano/spamd/spam as well as chmod g+w /var/lib/kopano/spamd/ham and it should be working.

                      P.S. Having the .eml files world read and writable seems no to be optimal :-)

                      BMWfan 1 Reply Last reply Reply Quote 0
                      • BMWfan
                        BMWfan @ashceryth last edited by

                        @ashceryth but inotify-spamlearn removes them already. I stopped only the process thats why i could send you the commandline output.

                        1 Reply Last reply Reply Quote 0
                        • ashceryth
                          ashceryth last edited by

                          @BMWfan

                          Hm, that’s strange. The log messages indicate that the files can’t be removed. Are you sure inotify-spamlearn is currently running as amavis? Are you still getting the errors?

                          1 Reply Last reply Reply Quote 0
                          • BMWfan
                            BMWfan last edited by BMWfan

                            @ashceryth sorry. You are right. I’ve executed now the two suggested lines.

                            chmod g+w /var/lib/kopano/spamd/ham
                            chmod g+w /var/lib/kopano/spamd/ham
                            

                            Depending on this failures

                            Jan 15 17:56:33 server inotify-spamlearn.py[19673]: bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.server.serverprofi24.de.19887 for /var/lib/spamassassin/bayes.lock: Permission denied
                            Jan 15 17:56:33 server inotify-spamlearn.py[19673]: plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.server.server.de.19887 for /var/lib/spamassassin/bayes.lock: Permission denied
                            Jan 15 17:56:33 server inotify-spamlearn.py[19673]: ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 500.
                            

                            i executed the following:

                            cd /var/lib/
                            chmod -R 777 spamassassin
                            

                            The kopano-spamd sa_group is nevertheless kopano and the inotify-spamlearn service is running as amavis user and amavis group. But it seems to work:

                            /var/log/kopano/spamd.log

                            2019-01-15 18:22:18,025 - spamd - INFO - Learning message as SPAM, entryid: 00000000A497753E7B1B4CE3894BB06ABB7C1F45010000000500000003DDF8D9711C4B69951C3DFB80B1E72E00000000
                            2019-01-15 18:22:26,102 - spamd - INFO - Learning message as SPAM, entryid: 00000000A497753E7B1B4CE3894BB06ABB7C1F450100000005000000FB5BCDFF29C04459B703EE1DB71C683500000000
                            2019-01-15 18:22:33,175 - spamd - INFO - Learning message as SPAM, entryid: 00000000A497753E7B1B4CE3894BB06ABB7C1F45010000000500000081D6E0E8508D4CF28F88A973162719C100000000
                            

                            /var/log/syslog

                            Jan 15 18:22:23 server inotify-spamlearn.py[20069]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/F80C680BD83440B8AA34556506BDC63D.eml: Learned tokens from 1 message(s) (1 message(s) examined)
                            Jan 15 18:22:23 server inotify-spamlearn.py[20069]: INFO Removing file: /var/lib/kopano/spamd/spam/F80C680BD83440B8AA34556506BDC63D.eml
                            Jan 15 18:22:30 server inotify-spamlearn.py[20069]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/CB8A83041D954CCB873E060165292F36.eml: Learned tokens from 1 message(s) (1 message(s) examined)
                            Jan 15 18:22:30 server inotify-spamlearn.py[20069]: INFO Removing file: /var/lib/kopano/spamd/spam/CB8A83041D954CCB873E060165292F36.eml
                            Jan 15 18:22:37 server inotify-spamlearn.py[20069]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/305A9D2637324400900B051A263CE4E3.eml: Learned tokens from 1 message(s) (1 message(s) examined)
                            Jan 15 18:22:37 server inotify-spamlearn.py[20069]: INFO Removing file: /var/lib/kopano/spamd/spam/305A9D2637324400900B051A263CE4E3.eml
                            

                            I want to let inotify-spamlearn log into the /var/log/kopano/spamd.log file, that’s why i changed in /etc/kopano/inotify-spamlearn.cfg this line:

                            # empty means log to console, use journalctl to read the logs when run from systemd service
                            logfile = /var/log/kopano/spamd.log
                            

                            I put amavis in the kopano group with gpasswd -a amavis kopano but it’s still not functioning:

                            Jan 15 18:54:19 server inotify-spamlearn.py[20669]: PermissionError: [Errno 13] Permission denied: '/var/log/kopano/spamd.log'
                            
                            $ /var/log/kopano# ls -la
                            total 80
                            drwxr-x---  2 kopano kopano 4096 Jan 15 18:55 .
                            drwxr-xr-x 11 root   root   4096 Jan 15 02:16 ..
                            -rw-r--r--  1 kopano kopano  334 Jan 15 05:19 dagent.log
                            -rw-r--r--  1 kopano kopano  530 Jan 13 08:31 dagent.log.1
                            -rw-r--r--  1 kopano kopano  254 Jan  4 21:23 dagent.log.2.gz
                            -rw-r--r--  1 kopano kopano  237 Dec 30 20:41 dagent.log.3.gz
                            -rw-r--r--  1 kopano kopano 7393 Jan 15 17:49 server.log
                            -rw-r--r--  1 kopano kopano 8225 Jan 13 17:31 server.log.1
                            -rw-r--r--  1 kopano kopano  420 Jan  5 20:08 server.log.2.gz
                            -rw-r--r--  1 kopano kopano 3575 Dec 30 20:40 server.log.3.gz
                            -rw-r--r--  1 kopano kopano 1229 Dec 22 19:43 server.log.4.gz
                            -rw-r--r--  1 kopano kopano 9996 Jan 15 18:22 spamd.log
                            -rw-r--r--  1 kopano kopano 1148 Jan  5 20:08 spamd.log.2.gz
                            -rw-r--r--  1 kopano kopano  530 Dec 30 12:10 spamd.log.3.gz
                            -rw-r--r--  1 kopano kopano  470 Dec 28 11:39 spamd.log.4.gz
                            

                            do you have a idea how i could fix this?

                            ashceryth 1 Reply Last reply Reply Quote 0
                            • ashceryth
                              ashceryth @BMWfan last edited by

                              Hi

                              @BMWfan said in Define ham and spam for spamassassin:

                              cd /var/lib/
                              chmod -R 777 spamassassin
                              

                              Every time you do this a puppy dies. In my humble opinion setting permissions to 777 is an absolute no-no. Your command makes the directory and everything below world read and writable. Are you sure you want that? :-)

                              do you have a idea how i could fix this?

                              The group has no write permission on the log file.

                              chmod g+w spamd.log

                              BMWfan 1 Reply Last reply Reply Quote 0
                              • BMWfan
                                BMWfan @ashceryth last edited by BMWfan

                                @ashceryth thanks for your help and your honestly opinion.
                                now, the inotifty-spamlearn log entries are in the /var/log/kopano/spamd.log

                                I checked the rights before i set chmod 777:

                                /var/lib/spamassassin# ls -la
                                total 24
                                drwxr-xr-x  6 debian-spamd debian-spamd 4096 Jan  8 20:05 .
                                drwxr-xr-x 50 root         root         4096 Jan  5 20:06 ..
                                drwx------  3 debian-spamd debian-spamd 4096 Dec 20 14:00 .spamassassin
                                drwxr-xr-x  6 debian-spamd debian-spamd 4096 Jan 15 03:04 3.004002
                                -rw-r--r--  1 root         root            0 Jan  8 19:55 bayes
                                drwxr-xr-x  3 debian-spamd debian-spamd 4096 Dec 20 14:01 compiled
                                drwx------  3 debian-spamd debian-spamd 4096 Dec 20 14:00 sa-update-keys
                                

                                I set now all rights back to this ^^.
                                How and where could i set the permission that a file like in this example called bayes.lock.server.server.de.19887 under /var/lib/spamassassin/ has to be created by a user (amavis) which is in the same group but not the owner of this folder in which the file has to be created? I think the name of the file is dynamic assigned why i can not create a file via touch and assign this file the needed rights.

                                1 Reply Last reply Reply Quote 0
                                • ashceryth
                                  ashceryth last edited by

                                  Hi @BMWfan

                                  The directory is owned by user and group debian-spamd. As I am not using Amavis/SpamAssassin I cannot check myself - but is user amavis member of group debian-spamd? If that’s the case you just have to fix the group permissions. However, I suggest not to mess too much with permissions.

                                  1 Reply Last reply Reply Quote 0
                                  • El Muchacho
                                    El Muchacho last edited by

                                    Took a while to figure out the right settings for my debian9 …
                                    But now all works as expected.

                                    My config for inotify-spamlearn:

                                    in /etc/systemd/inotify-spamlearn.service set

                                    run_as_group = amavis
                                    

                                    now set the rights for spam and ham folders

                                    chown -R kopano:amavis /var/lib/kopano/spamd
                                    

                                    and in order to delete the learned mails it’s neccessary that group amavis has the rights to delete the learned mails

                                    chmod g+w /var/lib/kopano/spamd/spam
                                    chmod g+w /var/lib/kopano/spamd/ham
                                    

                                    finally we need to reload the daemons for systemctl

                                    systemctl daemons-reload
                                    

                                    and restart the service inotify-spamlearn

                                    systemctl restart inotify-spamlearn
                                    

                                    that’s all ;)

                                    thctlo 1 Reply Last reply Reply Quote 0
                                    • thctlo
                                      thctlo @El Muchacho last edited by thctlo

                                      @El-Muchacho

                                      Thank you for the info.

                                      chown -R kopano:amavis /var/lib/kopano/spamd
                                      and
                                      chmod g+w /var/lib/kopano/spamd/spam
                                      chmod g+w /var/lib/kopano/spamd/ham

                                      in one liner :
                                      install -o kopano -g amavis -m 2770 -d /var/lib/kopano/spamd -d /var/lib/kopano/spamd/spam -d /var/lib/kopano/spamd/ham

                                      Does the “World” need read access in the spamd folder ?

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post