Navigation

    Kopano
    • Register
    • Login
    • Search
    • Categories
    • Get Official Kopano Support
    • Recent
    Statement regarding the closure of the Kopano community forum and the end of the community edition

    Define ham and spam for spamassassin

    Kopano Groupware Core
    5
    29
    4139
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • fbartels
      fbartels Kopano last edited by

      Best guess: the instructions are missing a reload of the unit files (after you copied the systemd unit)

      Regards Felix

      Resources:
      https://kopano.com/blog/how-to-get-kopano/
      https://documentation.kopano.io/
      https://kb.kopano.io/

      Support overview:
      https://kopano.com/support/

      BMWfan 1 Reply Last reply Reply Quote 0
      • BMWfan
        BMWfan @fbartels last edited by

        @fbartels said in Define ham and spam for spamassassin:

        reload of the unit files (after you copied the systemd unit)

        What do you mean with that? Or how should i proceed to do this?
        A restart of kopano-core and kopano-spamd is already done without any positive progress.

        Best Regards

        Daniel

        1 Reply Last reply Reply Quote 0
        • fbartels
          fbartels Kopano last edited by

          Restarting any Kopano service won’t here you, since systemd is complaining it cannot find a service unit that you have copied from a third party git repo. What I meant with the “reloading” is that systemctl usually complains that you have to execute a reload command, when you manually installed a unit file. But if this is not displayed to you then my second best guess is that you copied the unit file to the wrong location.

          Regards Felix

          Resources:
          https://kopano.com/blog/how-to-get-kopano/
          https://documentation.kopano.io/
          https://kb.kopano.io/

          Support overview:
          https://kopano.com/support/

          BMWfan 1 Reply Last reply Reply Quote 0
          • BMWfan
            BMWfan @fbartels last edited by BMWfan

            @fbartels said in [But if this is not displayed to you then my second best guess is that you copied the unit file to the wrong location.

            No, this is not displayed.

            I downloaded the files via wget and copied it in the folder as explained in the manual.

            /etc/systemd/system
            inotify-spamlearn.service
            
            /usr/local/sbin
            inotify-spamlearn.py
            
            /etc/kopano
            inotify-spamlearn.cfg
            

            then i tried to execute the command

            /etc/systemd/system# systemctl enable inotify-spamlearn.service
            Failed to enable unit: File inotify-spamlearn.service: Invalid argument
            

            tried to start it:

            /etc/systemd/system# systemctl start inotify-spamlearn.service
            Failed to start inotify-spamlearn.service: Unit inotify-spamlearn.service is not loaded properly: Invalid argument.
            See system logs and 'systemctl status inotify-spamlearn.service' for details.
            
            sudo systemctl status inotify-spamlearn
            ● inotify-spamlearn.service
               Loaded: error (Reason: Invalid argument)
               Active: inactive (dead)
            

            So, it should not be dependend on a wrong location, or what do you mean @fbartels ?

            Best Regards

            Daniel

            1 Reply Last reply Reply Quote 0
            • fbartels
              fbartels Kopano last edited by

              This works for me (Debian 9):

              root@system ~/inotify-spamlearn
              $ sudo cp inotify-spamlearn.service /etc/systemd/system
              
              root@system ~/inotify-spamlearn
              $ sudo systemctl status inotify-spamlearn.service
              ● inotify-spamlearn.service - Inotify Spamlearn
                 Loaded: loaded (/etc/systemd/system/inotify-spamlearn.service; disabled; vendor preset: enabled)
                 Active: inactive (dead)
              

              Regards Felix

              Resources:
              https://kopano.com/blog/how-to-get-kopano/
              https://documentation.kopano.io/
              https://kb.kopano.io/

              Support overview:
              https://kopano.com/support/

              BMWfan 1 Reply Last reply Reply Quote 0
              • BMWfan
                BMWfan @fbartels last edited by BMWfan

                @fbartels ok, it seems to be that the download was not complete of the config files thats why i had this issues. Now i’am stucking at the next problem.
                I have now all files again downloaded and replaced and gave the file /usr/local/sbin/inotify-spamlearn.py the following rights: rwx—r-x (User:root, Group: root) i hope this is correct.

                I installed via pip the python module inotify

                pip install inotify
                Collecting inotify
                  Using cached https://files.pythonhosted.org/packages/c7/fc/9728f1f708ecd5981007abe133d44fdcddf40915f8d13e12a140b77376ae/inotify-0.2.10-py2-none-any.whl
                Requirement already satisfied: nose in /usr/local/lib/python2.7/dist-packages (from inotify) (1.3.7)
                Installing collected packages: inotify
                Successfully installed inotify-0.2.10
                

                and tried to start inotify-spamlearn which gaves me no output.
                But, if it try to check the status i see this:

                sudo systemctl status inotify-spamlearn
                ● inotify-spamlearn.service - Inotify Spamlearn
                   Loaded: loaded (/etc/systemd/system/inotify-spamlearn.service; enabled; vendor preset: enabled)
                   Active: failed (Result: exit-code) since Sat 2018-12-29 13:57:46 CET; 6s ago
                  Process: 7172 ExecStart=/usr/local/sbin/inotify-spamlearn.py (code=exited, status=1/FAILURE)
                 Main PID: 7172 (code=exited, status=1/FAILURE)
                
                Dec 29 13:57:46 euve264608 inotify-spamlearn.py[7172]: Traceback (most recent call last):
                Dec 29 13:57:46 euve264608 inotify-spamlearn.py[7172]:   File "/usr/local/sbin/inotify-spamlearn.py", line 11, in <module>
                Dec 29 13:57:46 euve264608 inotify-spamlearn.py[7172]:     import inotify.adapters
                Dec 29 13:57:46 euve264608 inotify-spamlearn.py[7172]: ImportError: No module named 'inotify'
                

                This is the line 11

                import inotify.adapters
                

                As you can see here, the module is installed:

                pip list
                Package         Version
                --------------- -------
                configparser    3.5.0
                inotify         0.2.10
                logging         0.4.9.6
                MAPI            8.7.80
                nose            1.3.7
                pip             18.1
                PyICU           1.9.5
                python-dateutil 2.5.3
                pytz            2016.7
                pyzor           1.0.0
                setuptools      33.1.1
                six             1.10.0
                

                Do you have any idea how i could fix this?

                Best Regards

                Daniel

                1 Reply Last reply Reply Quote 0
                • fbartels
                  fbartels Kopano last edited by

                  No, sorry not familiar with this script.

                  Regards Felix

                  Resources:
                  https://kopano.com/blog/how-to-get-kopano/
                  https://documentation.kopano.io/
                  https://kb.kopano.io/

                  Support overview:
                  https://kopano.com/support/

                  BMWfan 1 Reply Last reply Reply Quote 0
                  • BMWfan
                    BMWfan @fbartels last edited by BMWfan

                    @fbartels thanks, nevertheless.

                    i found now the issue.

                    I have 2 python versions installed

                    $ python --version
                    Python 2.7.13
                    
                    $ python3 --version
                    Python 3.5.3
                    

                    As i explained before, i have installed the needed module via pip install inotify which install it for python 2.7

                    Requirement already satisfied: nose in /usr/local/lib/python2.7/dist-packages (from inotify) (1.3.7)
                    

                    The requirements as i see in the python script inotify-spamlearn.py in the first line are amongst other things python 3

                    #!/usr/bin/env python3
                    

                    So i had to find a way to install the needed module for python 3 what ive done like this:

                    $ apt-get install python3-setuptools
                    $ easy_install3 pip
                    $ pip-3.5 install inotify
                    

                    it works:

                    $ systemctl status inotify-spamlearn.service
                    ● inotify-spamlearn.service - Inotify Spamlearn
                       Loaded: loaded (/etc/systemd/system/inotify-spamlearn.service; enabled; vendor preset: enabled)
                       Active: active (running) since Tue 2019-01-01 16:03:38 CET; 7s ago
                     Main PID: 1062 (python3)
                       CGroup: /system.slice/inotify-spamlearn.service
                               └─1062 python3 /usr/local/sbin/inotify-spamlearn.py
                    
                    Jan 01 16:03:38 euve264608 inotify-spamlearn.py[1062]: INFO Starting inotify-spamlearn.py
                    Jan 01 16:03:38 euve264608 inotify-spamlearn.py[1062]: INFO Looking for existing files in /var/lib/kopano/spamd/spam
                    Jan 01 16:03:38 euve264608 inotify-spamlearn.py[1062]: INFO Finished looking for existing files in /var/lib/kopano/spamd/spa
                    Jan 01 16:03:38 euve264608 inotify-spamlearn.py[1062]: INFO Looking for existing files in /var/lib/kopano/spamd/ham
                    Jan 01 16:03:38 euve264608 inotify-spamlearn.py[1062]: INFO Finished looking for existing files in /var/lib/kopano/spamd/ham
                    Jan 01 16:03:38 euve264608 inotify-spamlearn.py[1062]: INFO Inotify learning started
                    

                    Best Regards

                    Daniel

                    1 Reply Last reply Reply Quote 0
                    • BMWfan
                      BMWfan last edited by BMWfan

                      @fbartels perhaps you have a idea here. As you can see in the post before that my inotify-spamlearn is running.
                      If i try to mark a mail as spam in the webapp dashboard then i got the following output:

                      Jan 01 16:35:56 server inotify-spamlearn.py[1367]: INFO Inotify learning started
                      Jan 01 16:36:55 server inotify-spamlearn.py[1367]: config: path "/dev/null/.spamassassin" is inaccessible: Not a directory
                      Jan 01 16:36:55 server inotify-spamlearn.py[1367]: config: path "/dev/null/.spamassassin/user_prefs" is inaccessible: Not a directory
                      Jan 01 16:36:56 server inotify-spamlearn.py[1367]: config: path "/dev/null/.spamassassin" is inaccessible: Not a directory
                      Jan 01 16:36:56 server inotify-spamlearn.py[1367]: config: path "/dev/null/.spamassassin" is inaccessible: Not a directory
                      Jan 01 16:36:56 server inotify-spamlearn.py[1367]: bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /dev/null/.spamassassin/bayes.lock.server.server.1434 for /dev/null/.spamassassin/bayes.lock: Not a directory
                      Jan 01 16:36:56 server inotify-spamlearn.py[1367]: plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /dev/null/.spamassassin/bayes.lock.server.server.1434 for /dev/null/.spamassassin/bayes.lock: Not a directory
                      Jan 01 16:36:56 server inotify-spamlearn.py[1367]: ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 500.
                      Jan 01 16:36:56 server inotify-spamlearn.py[1367]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/0A8EFD57433D41F983C4ED716DA7CE5D.eml: Learned tokens from 0 message(s) (1 message(s) examined)
                      Jan 01 16:36:56 server inotify-spamlearn.py[1367]: INFO Removing file: /var/lib/kopano/spamd/spam/0A8EFD57433D41F983C4ED716DA7CE5D.eml
                      

                      There is a path ~/.spamassassin which has amongst other things these file included:

                      root@system:~/.spamassassin# ls
                      user_prefs
                      

                      But under /dev is null a file and not a directory

                      root@system:/dev# cd null
                      -bash: cd: null: Not a directory
                      

                      Best Regards

                      Daniel

                      1 Reply Last reply Reply Quote 0
                      • ashceryth
                        ashceryth last edited by ashceryth

                        Hi @BMWfan

                        I think this is because the default home of the kopano user is /dev/null (you can check your /etc/passwd) and inotify-spamlearn is running as kopano user. It seems like ~/.spamassassin/ (where ~/ is the home directory of the kopano user - /dev/null) is the default directory used by the sa-learn command. You can specify the paths used by the sa-learn command in inotify-spamlearn.cfg or run inotify-spamlearn as different user (set in the systemd unit file).

                        Regards
                        Patrick

                        BMWfan 1 Reply Last reply Reply Quote 0
                        • BMWfan
                          BMWfan @ashceryth last edited by BMWfan

                          @ashceryth thanks for your response. I’am thinking about it what would be the best choice todo.
                          I’am not sure if it would help and would be good if i assign the kopano user a home directory, because /dev/null is standard if you dont set once i think. What have you done to get it work?

                          For kopano-spamd i assigned already kopano to the amavis group

                          gpasswd -a kopano amavis
                          
                          1 Reply Last reply Reply Quote 0
                          • ashceryth
                            ashceryth last edited by ashceryth

                            I suggest to try to run inotify-spamlearn as the same user as amavis runs as.

                            So, edit the systemd unit file (/etc/systemd/system/inotify-spamlearn.service) and set:

                            User=amavis
                            Group=amavis
                            

                            Run systemctl daemon-reload after editing and restart the service with systemctl restart inotify-spamlearn.

                            Check your spamd.cfg to make sure it uses the amavis group (the .eml files should then get the gid of the amavis group):

                            sa_group = amavis

                            Does it work then?

                            I don’t use the sa-learn command. I use Rspamd and let inotify-spamlearn send the mails via HTTP post requests with curl.

                            BMWfan 1 Reply Last reply Reply Quote 0
                            • BMWfan
                              BMWfan @ashceryth last edited by BMWfan

                              @ashceryth said in Define ham and spam for spamassassin:

                              systemctl restart inotify-spamlearn

                              No, this solves it not. The problem is after i change the sa_group at /etc/kopano/spamd.cfg to amavis then i get permission errors in the kopano-spamd process as here described: Kopano spamd error - PermissionError: [Errno 1] Operation not permitted

                              My configuration is now this:
                              /etc/kopano/spamd.cfg

                              # run as specific user
                              run_as_user         = kopano
                              
                              # run as specific group
                              run_as_group        = kopano
                              
                              # Spamassassin group
                              sa_group = kopano
                              

                              /etc/systemd/system/inotify-spamlearn.service

                              [Service]
                              User=kopano
                              Group=kopano
                              

                              but then again with this log entries:

                              Jan 12 16:13:10 euve264608 inotify-spamlearn.py[7348]: config: path "/dev/null/.spamassassin" is inaccessible: Not a directory
                              Jan 12 16:13:10 euve264608 inotify-spamlearn.py[7348]: config: path "/dev/null/.spamassassin/user_prefs" is inaccessible: Not a directory
                              Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.euve264608.serverprofi24.de.7427 for /var/lib/spamassassin/bayes.lock: Permission denied
                              Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.euve264608.serverprofi24.de.7427 for /var/lib/spamassassin/bayes.lock: Permission denied
                              Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 500.
                              Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/72C00ABE4488476D8AF626120D868211.eml: Learned tokens from 0 message(s) (1 message(s) examined)
                              Jan 12 16:13:14 euve264608 inotify-spamlearn.py[7348]: INFO Removing file: /var/lib/kopano/spamd/spam/72C00ABE4488476D8AF626120D868211.eml
                              

                              if i change only these entries from kopano to amavis
                              /etc/systemd/system/inotify-spamlearn.service

                              [Service]
                              User=amavis
                              Group=amavis
                              

                              then i get this messages:

                              Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Starting inotify-spamlearn.py
                              Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Looking for existing files in /var/lib/kopano/spamd/spam
                              Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Finished looking for existing files in /var/lib/kopano/spamd/spam
                              Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Looking for existing files in /var/lib/kopano/spamd/ham
                              Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Inotify learning started
                              Jan 12 16:20:41 Servername inotify-spamlearn.py[7501]: INFO Finished looking for existing files in /var/lib/kopano/spamd/ham
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.Servername.serverprofi24.de.7506 for /var/lib/spamassassin/bayes.lock: Permission denied
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/lib/spamassassin/bayes.lock.Servername.serverprofi24.de.7506 for /var/lib/spamassassin/bayes.lock: Permission denied
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: ERROR: the Bayes learn function returned an error, please re-run with -D for more information at /usr/bin/sa-learn line 500.
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml: Learned tokens from 0 message(s) (1 message(s) examined)
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: INFO Removing file: /var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: Exception in thread Inotify Handling:
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: Traceback (most recent call last):
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 80, in inotified
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     process('/'.join([watch_path, filename]), spamcmd, delete, initiator)
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 43, in process
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     os.remove(filename)
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: PermissionError: [Errno 13] Permission denied: '/var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml'
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: During handling of the above exception, another exception occurred:
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: Traceback (most recent call last):
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     self.run()
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/lib/python3.5/threading.py", line 862, in run
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     self._target(*self._args, **self._kwargs)
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 85, in inotified
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     i.remove_watch([spam_dir, ham_dir])
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/lib/python3.5/dist-packages/inotify/adapters.py", line 109, in remove_watch
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     wd = self.__watches.get(path)
                              Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: TypeError: unhashable type: 'list'
                              

                              These are the amavis and kopano rights under /etc/groups

                              amavis:x:123:clamav,kopano
                              kopano:x:999:kapi,konnect,amavis
                              
                              ashceryth 1 Reply Last reply Reply Quote 0
                              • ashceryth
                                ashceryth last edited by ashceryth

                                @BMWfan said in Define ham and spam for spamassassin:

                                No, this solves it not. The problem is after i change the sa_group at /etc/kopano/spamd.cfg to amavis then i get permission errors in the kopano-spamd process as here described: Kopano spamd error - PermissionError: [Errno 1] Operation not permitted

                                Maybe because the problem described by @fbartels in this post insn’t solved for you yet?

                                Who is the owner/group and what are the permissions of the files in /var/lib/kopano/spamd/spam/ (just ls -la)?

                                And please test the sa-learn command manually as the desired user before configuring it with inotify-spamlearn to see if everything works as expected.

                                BMWfan 1 Reply Last reply Reply Quote 0
                                • BMWfan
                                  BMWfan @ashceryth last edited by

                                  @ashceryth thanks for your reponse and hints.
                                  Here is the suggested output:

                                  $ /var/lib/kopano/spamd/spam# ls -la
                                  total 8
                                  drwxr-xr-x 2 kopano kopano 4096 Jan 12 17:05 .
                                  drwxr-xr-x 4 kopano kopano 4096 Dec 27 10:32 ..
                                  
                                  1 Reply Last reply Reply Quote 0
                                  • ashceryth
                                    ashceryth @BMWfan last edited by

                                    Well, at the moment your spam folder is empty. Normally the .eml files get deleted after learning by inotify-spamlearn. However, according to your log from above it wasn’t possible to remove them because of permission issues:

                                    @BMWfan said in Define ham and spam for spamassassin:

                                    Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:   File "/usr/local/sbin/inotify-spamlearn.py", line 43, in process
                                    Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]:     os.remove(filename)
                                    Jan 12 16:21:00 Servername inotify-spamlearn.py[7501]: PermissionError: [Errno 13] Permission denied: '/var/lib/kopano/spamd/spam/A5D6C47FA96743C1B2DA0FE7203DA793.eml'
                                    

                                    That’s why I’m asking. Would be interesting to see the permissions of the .eml files themselves.

                                    BMWfan 1 Reply Last reply Reply Quote 0
                                    • BMWfan
                                      BMWfan @ashceryth last edited by BMWfan

                                      @ashceryth i stopped in the inotify-spamlearn service and here is the suggested output:

                                      $ /var/lib/kopano/spamd/spam# ls -la
                                      total 12
                                      drwxr-xr-x 2 kopano kopano 4096 Jan 14 20:39 .
                                      drwxr-xr-x 4 kopano kopano 4096 Dec 27 10:32 ..
                                      -rw-rw-rw- 1 kopano kopano 3643 Jan 14 20:39 600F5F44181B4F2B865EAE2A45EC2573.eml
                                      
                                      1 Reply Last reply Reply Quote 0
                                      • ashceryth
                                        ashceryth last edited by

                                        The reason for the permission error messages is that your spam directory is not writable for the group. Without write permissions the .eml files cannot be removed by inotify-spamlearn running as amavis (member of kopano group).

                                        Just run chmod g+w /var/lib/kopano/spamd/spam as well as chmod g+w /var/lib/kopano/spamd/ham and it should be working.

                                        P.S. Having the .eml files world read and writable seems no to be optimal :-)

                                        BMWfan 1 Reply Last reply Reply Quote 0
                                        • BMWfan
                                          BMWfan @ashceryth last edited by

                                          @ashceryth but inotify-spamlearn removes them already. I stopped only the process thats why i could send you the commandline output.

                                          1 Reply Last reply Reply Quote 0
                                          • ashceryth
                                            ashceryth last edited by

                                            @BMWfan

                                            Hm, that’s strange. The log messages indicate that the files can’t be removed. Are you sure inotify-spamlearn is currently running as amavis? Are you still getting the errors?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post