• Hi@all,

    i use Kopano on a UCS member server (4.4-6 errata803) Kopano is installed in the versions

    WebApp: 4.6.3.0+155.1
    Kopano Core: 8.7.16
    Z-Push: 2.6.0+0-0
    

    For the setup of kopano-spamd I followed these hints:

    https://documentation.kopano.io/kopanocore_administrator_manual/special_kc_configurations.html#configuring-kopano-spamd-for-automatic-spam-ham-learning

    https://github.com/bkram/inotify-spamlearn

    My steps of the installation:

    univention-install kopano-spamd
    usermod -aG amavis kopano
    univention-install python3-pip
    pip3 install inotify
    cp /etc/kopano/spamd.cfg /etc/kopano/spamd.cfg.org
    

    Edit /etc/kopano/spamd.cfg:

    run_as_group = amavis
    learn_ham = yes
    

    The entire file:

    # run as specific user
    #run_as_user         = amavis
    # run as specific group
    run_as_group        = amavis
    # control pid file
    #pid_file            =   /var/run/kopano/spamd.pid
    # run server in this path (when not using the -F switch)
    #running_path = /var/lib/kopano
    # LOG SETTINGS
    # Logging method (syslog, file)
    log_method          =   syslog
    # Loglevel (0(none), 1(crit), 2(err), 3(warn), 4(notice), 5(info), 6(debug))
    #log_level           =   3
    # Logfile for log_method = file, use '-' for stderr
    #log_file            =   /var/log/kopano/spamd.log
    # Log timestamp - prefix each log line with timestamp in 'file' logging mode
    #log_timestamp       =   1
    #spam_dir = /var/lib/kopano/spamd/spam
    #spam_db = /var/lib/kopano/spamd/spam.db
    # Learn ham, when the user moves emails from junk to inbox,
    # enabled by default.
    learn_ham = yes
    # The dir where ham mails are written to which are later picked up
    # by the sa-learn program
    #ham_dir = /var/lib/kopano/spamd/ham
    # Spamassassin group
    sa_group = amavis
    # Header tag for spam emails
    #header_tag = X-Spam-Flag
    

    Files download from https://github.com/bkram/inotify-spamlearn:

    [inotify-spamlearn.cfg]

    [paths]
    # path to look for spam emls
    spam_dir = /var/lib/kopano/spamd/spam
    # path to look for ham emls
    ham_dir = /var/lib/kopano/spamd/ham
    
    [spam]
    # command to pipe the spam into
    spamcmd = /usr/bin/sa-learn --spam
    # command to pipe the ham into
    hamcmd = /usr/bin/sa-learn --ham
    
    [mode]
    # should we delete emls after processing
    delete = True
    # should we scan and process existing spam/ham in the watch directories
    scan = True
    # after scanning and processing of existing spam/ham exit the script. (crontab mode)
    oneshot = False
    
    [logging]
    # levels possible are DEBUG INFO WARN ERROR CRITICAL
    loglevel = INFO
    # empty means log to console, use journalctl to read the logs when run from systemd service
    logfile =
    

    [inotify-spamlearn.service]

    [Unit]
    Description=Inotify Spamlearn
    
    [Service]
    User=amavis
    Group=amavis
    Type=simple
    ExecStart= /usr/local/sbin/inotify-spamlearn.py
    ExecReload=/bin/kill -HUP $MAINPID
    
    [Install]
    WantedBy=multi-user.target
    
    

    I have not modified the file inotify-spamlearn.py

    The next steps:

    cp inotify-spamlearn.cfg /etc/kopano
    cp inotify-spamlearn.py /usr/local/sbin/
    cp inotify-spamlearn.service /etc/systemd/system
    chmod a+x /usr/local/sbin/inotify-spamlearn.py
    chown -R kopano:amavis /var/lib/kopano/spamd
    chmod g+w /var/lib/kopano/spamd/spam
    chmod g+w /var/lib/kopano/spamd/ham
    systemctl daemon-reload
    
    • Activate and start *
    systemctl enable inotify-spamlearn
    systemctl start inotify-spamlearn
    
    • Check status *
    root@com01:~# systemctl status inotify-spamlearn
    ● inotify-spamlearn.service - Inotify Spamlearn
       Loaded: loaded (/etc/systemd/system/inotify-spamlearn.service; enabled; vendor preset: enabled)
       Active: active (running) since Wed 2020-11-18 08:49:44 CET; 50min ago
     Main PID: 818 (python3)
        Tasks: 2 (limit: 4915)
       Memory: 11.5M
          CPU: 338ms
       CGroup: /system.slice/inotify-spamlearn.service
               └─818 python3 /usr/local/sbin/inotify-spamlearn.py
    
    Nov 18 08:49:44 com01 systemd[1]: Started Inotify Spamlearn.
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Starting inotify-spamlearn.py
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Looking for existing files in /var/lib/kopano/spamd/spam
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Finished looking for existing files in /var/lib/kopano/spamd/spam
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Looking for existing files in /var/lib/kopano/spamd/ham
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Inotify learning started
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Finished looking for existing files in /var/lib/kopano/spamd/ham
    

    to see what happens if I mark a mail as spam in the inbox and as HAM in the junk folder

    journalctl -u inotify-spamlearn -f
    
    • Example 1 *
    • I get about 3-6 spam mails a day. These always come from the same sender. But only in the field “FROM” there is always the same sender. But if you look at the mail in detail, there is a different address in the sender address each time.

    Here is the header of the file as it arrived today:

    Return-Path: <de@have.pecces.com>
    Received: from com01.domain.internal (127.0.0.1:55520)
    	by com01 (kopano-dagent) with LMTP;
    	Sun, 15 Nov 2020 08:06:45 +0100 (CET)
    Received: from localhost (localhost [127.0.0.1])
    	by com01.domain.internal (Postfix) with ESMTP id DC20EDE1CF
    	for <me@mydomain.de>; Sun, 15 Nov 2020 08:06:45 +0100 (CET)
    X-Virus-Scanned: by amavisd-new-2.10.1 (20141025) (Debian) at domain.internal
    X-Spam-Flag: NO
    X-Spam-Score: 0.011
    X-Spam-Level:
    X-Spam-Status: No, score=0.011 tagged_above=-1000 required=5
    	tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1,
    	HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.1, SPF_HELO_NONE=0.001,
    	SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01] autolearn=disabled
    Received: from com01.domain.internal ([127.0.0.1])
    	by localhost (com01.domain.internal [127.0.0.1]) (amavisd-new, port 10024)
    	with ESMTP id GEK0ILSyLIpX for <me@mydomain.de>;
    	Sun, 15 Nov 2020 08:06:44 +0100 (CET)
    Received: from com01.domain.internal (localhost [127.0.0.1])
    	by com01.domain.internal (Postfix) with ESMTP id A9C07DF703
    	for <me@mydomain.de>; Sun, 15 Nov 2020 08:06:44 +0100 (CET)
    X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
    	dd44626.provider.com
    X-Original-To: me.mueller@domain2.de.de
    Delivered-To: ispuser@dd44626.provider.com
    X-Greylist: delayed 7204 seconds by postgrey-1.34 at dd44626; Sun, 15 Nov 2020 08:06:02 CET
    Authentication-Results: dd44626.provider.com;
    	dkim=pass (1024-bit key; unprotected) header.d=pecces.com header.i=de@have.pecces.com header.b="X1w/g76p";
    	dkim-atps=neutral
    X-policyd-weight: using cached result; rate: -6.1
    Received: from w0193f8d.provider.com [11.22.33.44]
    	by com01.domain.internal with POP3 (fetchmail-6.3.26)
    	for <me@mydomain.de> (single-drop); Sun, 15 Nov 2020 08:06:44 +0100 (CET)
    Received: from host0.pecces.com (host0.pecces.com [146.59.199.89])
    	by dd44626.provider.com (Postfix) with ESMTPS id EDA0AC03B22
    	for <me.mueller@domain2.de.de>; Sun, 15 Nov 2020 08:06:02 +0100 (CET)
    DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=mailer; d=pecces.com;
     h=Date:To:From:Reply-To:Subject:Message-ID:List-Unsubscribe:MIME-Version:
     Content-Type; i=de@have.pecces.com;
     bh=O6bTbsinAi8Lljk8YJgeSknxbDoZwYWF+GArhTsbnsI=;
     b=X1w/g76piY0LOrA0n6hLqRgnIMsrLaRHMzvhun0NmWrea5DSgKtey6hBUAKL0tig6fowuuhK8odx
       KN3B2iNREQruXe0ar9ZsAE/ZYuGYfbwAjdrmoCmZVMOgDJ8aw/H4n5+gD9e6QXwdqcDP6vyZt4pF
       AHl/QyxMOYm/PmYRPYE=
    Date: Sun, 15 Nov 2020 13:05:56 +0800
    To: me.mueller@domain2.de.de
    From: Bellgs <de@have.pecces.com>
    Reply-To: Bellgs <de@have.pecces.com>
    Subject: =?UTF-8?Q?Etwas_muss_sich_=C3=A4ndern.?=
    Message-ID: <ecdcd70f2a15ac1442aa8ffd634096b7@iZ0xi729wodf0k0eja0uqoZ>
    X-Mailer: Sendy (https://sendy.co)
    List-Unsubscribe: <http://r.sconfit.com/sendy/unsubscribe/Kwq81Rnr6gb44OU8IIT4um2gzwpGQy4AxPJZQSAIBjo/lWs5rtSyFNAsAaeQVJfpUA/1UXUIFyeVsCYfvqXYIy892kQ>
    MIME-Version: 1.0
    Content-Type: text/html; charset=UTF-8
    X-KasLoop: ispuser
    
    <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
    <html lang="en">
    <head><meta name="ROBOTS" content="NOINDEX, NOFOLLOW"><meta name="referrer" content="no-referrer">
    	<title></title>
    	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta name="viewport" content="initial-scale=1.0"><!-- So that mobile webkit will display zoomed in --><meta name="format-detection" content="telephone=no"><!-- disable auto telephone linking in iOS -->
    </head>
    

    If I mark them as spam in Kopano the following happens on the server:

    -- Reboot --
    Nov 18 08:49:44 com01 systemd[1]: Started Inotify Spamlearn.
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Starting inotify-spamlearn.py
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Looking for existing files in /var/lib/kopano/spamd/spam
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Finished looking for existing files in /var/lib/kopano/spamd/spam
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Looking for existing files in /var/lib/kopano/spamd/ham
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Inotify learning started
    Nov 18 08:49:47 com01 inotify-spamlearn.py[818]: INFO Finished looking for existing files in /var/lib/kopano/spamd/ham
    Nov 18 09:58:05 com01 inotify-spamlearn.py[818]: INFO Processing [Inotify] /var/lib/kopano/spamd/spam/B20D94D163FC40BE869DABE867ED020F.eml: Learned tokens from 1 message(s) (1 message(s) examined)
    Nov 18 09:58:05 com01 inotify-spamlearn.py[818]: INFO Removing file: /var/lib/kopano/spamd/spam/B20D94D163FC40BE869DABE867ED020F.eml
    

    It looks to me like the spam filter uses this mail to learn. I mark these mails every day but the recognition does not get better.

    • Example 2 * but on a different server. Exactly the same UCS and Kopano version and exactly the same configuration of the components. This mail is a newsletter to which the user has subscribed.
    Return-Path: <musikblitz@musikpressedienst.de>
    Received: from com01.foobar.internal (127.0.0.1:34188)
    	by com01 (kopano-dagent) with LMTP;
    	Thu, 10 Sep 2020 09:48:53 +0200 (CEST)
    Received: from localhost (localhost [127.0.0.1])
    	by com01.foobar.internal (Postfix) with ESMTP id 2FF14803C55A
    	for <max.mustermann@company.de>; Thu, 10 Sep 2020 09:48:53 +0200 (CEST)
    X-Virus-Scanned: by amavisd-new-2.10.1 (20141025) (Debian) at foobar.internal
    X-Spam-Flag: YES
    X-Spam-Score: 5.211
    X-Spam-Level: *****
    X-Spam-Status: Yes, score=5.211 tagged_above=-1000 required=5
    	tests=[HDRS_LCASE_IMGONLY=0.1, HELO_MISC_IP=0.174,
    	HTML_IMAGE_ONLY_08=1.781, HTML_MESSAGE=0.001, RDNS_NONE=1.274,
    	SPF_NONE=0.001, TVD_SPACE_RATIO=0.001, TVD_SPACE_RATIO_MINFP=1.878,
    	URIBL_BLOCKED=0.001] autolearn=disabled
    Received: from com01.foobar.internal ([127.0.0.1])
    	by localhost (com01.foobar.internal [127.0.0.1]) (amavisd-new, port 10024)
    	with ESMTP id zmBU9lTRm3QS for <max.mustermann@company.de>;
    	Thu, 10 Sep 2020 09:48:49 +0200 (CEST)
    Received: from com01.foobar.internal (localhost [127.0.0.1])
    	by com01.foobar.internal (Postfix) with ESMTP id 2F18E801515C
    	for <max.mustermann@company.de>; Thu, 10 Sep 2020 09:48:49 +0200 (CEST)
    Received: from pop.provider.de [212.227.15.171]
    	by com01.foobar.internal with POP3 (fetchmail-6.3.26)
    	for <max.mustermann@company.de> (single-drop); Thu, 10 Sep 2020 09:48:49 +0200 (CEST)
    Received: from [217.72.192.67] ([217.72.192.67]) by mx.kundenserver.de
     (mxeue109 [217.72.192.67]) with ESMTPS (Nemesis) id 1MtvMg-1kWZFA35S8-00uGlT
     for <max.mustermann@company.de>; Thu, 10 Sep 2020 09:48:04 +0200
    Received: from mail6.melting-mind.de ([195.226.185.125]) by mx.kundenserver.de
     (mxeue109 [217.72.192.67]) with ESMTPS (Nemesis) id 1MHH8g-1kKh9A2yki-00DIXb
     for <max.mustermann@company-edition.de>; Thu, 10 Sep 2020 09:48:04
     +0200
    Received: from localhost (localhost [127.0.0.1])
    	by mail6.melting-mind.de (Postfix) with ESMTP id 045AA9A4C5F
    	for <max.mustermann@company-edition.de>; Thu, 10 Sep 2020 09:48:02 +0200 (CEST)
    X-Virus-Scanned: Debian amavisd-new at mail6.melting-mind.de
    Received: from mail6.melting-mind.de ([127.0.0.1])
    	by localhost (mail6.melting-mind.de [127.0.0.1]) (amavisd-new, port 10026)
    	with ESMTP id j-1cRs-Tl7fR
    	for <max.mustermann@company-edition.de>;
    	Thu, 10 Sep 2020 09:48:01 +0200 (CEST)
    Received: from [192.168.1.77] (p5de1c80b.dip0.t-ipconnect.de [93.225.200.11])
    	(Authenticated sender: blitz@musikpressedienst.de)
    	by mail6.melting-mind.de (Postfix) with ESMTPA id 9E4CA9A4C7A
    	for <max.mustermann@company-edition.de>; Thu, 10 Sep 2020 09:47:57 +0200 (CEST)
    Message-Id: <Dpt5N0WLhq0q4gUWfLgrA92V0tm2t2WRQSc7K6CJ9OcF@musikpressedienst.de>
    Mime-Version: 1.0
    From: musikblitz@musikpressedienst.de
    To: =?iso-8859-1?Q?"J=FCrgen_Rinschler"?= <max.mustermann@company-edition.de>
    Subject: musikpressedienst meldet
    Date: Thu, 10 Sep 2020 09:47:58 +0200
    X-Bounce-Tracking-Info: <SsO8cmdlbglSaW5zY2hsZXIJQmVsbGEgTXVzaWNhCWp1ZXJnZW4ucmluc2NobGVyQGJlbGxhLW11c2ljYS1lZGl0aW9uLmRlCW11c2lrcHJlc3NlZGllbnN0IG1lbGRldAk0ODYwCW11c2lrYmxpdHoJMTIxCWJvdW5jZQlubwlubw==>
    Content-type: multipart/alternative;
         Boundary="--=BOUNDARY_910947_JFCH_JLXH_HXPI_SCJI"
    Envelope-To: <max.mustermann@company.de>
    X-Spam-Flag: NO
    X-UI-Filterresults: notjunk:1;V03:K0:JvLY7sqQ6mU=:r8p0Ic1Ye3TvUIG8q3fbwThFnE
     y247+brhcqVkgfDhO1QXB6iy9i+xeCHqpqa7fvN8FGvl1ksim3QevEFtvpFJ+LKpH6FIuu0sO
     ycUV5lfOePcODRtT92tSl5fWFPhiBE5cVu7nmhwbNijGqGEnzQUpbkYuhGcw8IT4C8pgPsmbC
     S3oRrHvTSPqovQ8TRqH1o44nnbfDpbaZmNJ2/8CSWHtG4tIirQVxEKt3L88ZHh2tjJ92OOVK4
     QYvRPFq15ang1Opq4OtXAdoYJB1Ne6bMgCOBK5wHg4hBmoevl7nNGM3BXtkPr4iG/YYeGVVmS
     72NvUl1t36kepqQb8HAZqjL/V5KpPR3h6zyUR+Ywbx0hSBwRooxGWFHdR/Sw0BDFOByeQeqL6
     XA4I8WLVaEPIB4sOWTuOmVXkBb0C1D24k6cj0KeHGOBpMyErsBmaBrwWLpEOEwGiRbi5W90+s
     4sbYT5yV9IxFoBYJQKKejEthjRBsYU98o4xG+gQI2lSpFmQem7qmbSobbF428ZnETcYov7K8X
     HiTAxYr0YS1Atz0aCquUXLTezB8LtplnLuDoWl0wJ/CxQg9TvUTgCKXIQ0/1IyXOfbz48c8/o
     p1Fdns76klSbezzbtrof7Eg7KqMM+D9/UNRwUtfHz8gqhTBMAi/Mj+znXPZMG3RttG2c26lkZ
     jfwaop3dZPTcD/qAusGMBL5pa0lApNo/8zO6M3PqUSQLJZJACvJS+bzam/wS9EJ3lsEmXbv0C
     ETJ1a6FqZ4MaGY2NPoOqCfQI7miPrc7+u6SdRQ5c/lB6UuqZO/HjY03RgN5yFLPs4ymnXlq3I
     A+K21Dm98JELZy0m0ym9+Q3dzea4sC9lG2rAxKWIQCRX7va3m1dnp8sE06hEODDmK1Wb2ff/C
     a1QnQukhPl4sTlediDLA2tulTLuxVCJ/mJGCi56v8gb7aJK2pcbmiID+PWoVPRtXJYNM03lwk
     O10iJYPXhEkzW9RUzdsE831ItZRAlyCST87mAcZ3Q9YwjsgvDpQzDFEsxJs7QQ6hBjIbnYHis
     IC7Niq3HnUZrsEcgQHaBR6iPemrEoMiKlaCCDNQ+IF9dWAeIZz+i0Zc+CHLazt0VB40gWJt9r
     OtKfx+ArKldD9/URZ+Z+4uDunhZRE7S/+40Rw36iI/ta2sN+hJqvP/hhUWxhekWcKvVDj9YDM
     84AgT7dbeXGX4ZGiJr+WmsMDdx8oXyy4ZPZ9jib3qfIbr823fri5uZDygDv40/q8i7xM91Yk7
     4RQHDpFHBHSFLRa2uPdFxEIaB9iBvRoWpVZdp8acKzs85GZEd/9QOVC/T4T/RLvR1RrD4Irke
     ZVqwPbfEnMAzY8uG22X38iJASk+wuHJOLbfPmBlKsqYiAYP0kCKVLG630nhwoVP+pp+MM7sym
     LqZ4DRVFzFhHS2VepON/fyTpMqlTp/BQdTAv9llhccUmBl+Hj3eH+yfLDP4jMICBPfl7mm890
     nzOHB+yubumJiGDF3CxMihQEd75rleMYj5fKKRjt0IOLPLH37+gIpqOn8EwS3Or2ZIFwZ7l+j
     GpBLJ16SKTJQ86zCIZ+XcdcqvfKWa1eadqHS/GHfBzHIi7/vlaAFMdzc2sU3Ri2dHDaceF0I/
     aYOxJi2sJFQSHW4WyLBO8EhgqmjSMCJyFZ8UBUJp2iXp7IeD4fADjuhQas8xjqGnQFqZn/xhx
     p+aSdnOHWcBR3QPgOV/5/Y/IDNZ+JTvKpibp230owyAVkx02KiQeuIohI58/wMfSkcP9byv62
     p4MFTs6bWNL34L63fU2ne5R9P/coUNSuUI4nwMUk5XBAFOMSb47MG9OF5btk6oV1aVto67WCL
     RTywqgwO/69NQYX1JidOPc3ed5aHt42hpFlsob9JKPlxyoyEqhUWqnVpruF0A2CaeZu5CazTJ
     h2VrczVum9yBIQkKkvtYD9yv+o9S8nFFVbxQPT9coBo5cGISb3qjVFV7FZVopcPuqRdourcju
     byUnvcyZA==
    
    Die Nachricht ist im MIME-Format. Da Ihr E-Mail-Programm dieses Format nicht
    unterstützt, werden einzelne oder alle Nachrichten nicht lesbar sein.
    
    ----=BOUNDARY_910947_JFCH_JLXH_HXPI_SCJI
    Content-type: text/plain;
         charset=iso-8859-1; format=flowed
    Content-transfer-encoding: quoted-printable
    
    
    16=2ESeptember: Reeperbahn Festival startet - viel Programm
    http://mailings=2Einfernoevents=2Ecom/m/13704303/
    ----=BOUNDARY_910947_JFCH_JLXH_HXPI_SCJI
    Content-type: multipart/related;
         Boundary="--=BOUNDARY_910947_QDWV_HLHY_CYBN_XKVV"
    
    ----=BOUNDARY_910947_QDWV_HLHY_CYBN_XKVV
    Content-type: text/html;
         charset=iso-8859-1
    Content-transfer-encoding: quoted-printable
    

    When I move this mail from the Junk-Folder to the inbox nothing happens in the logfile.

    I have searched here in the forum but found no solution. What do I have to adjust to improve detection?

    with best
    pixel24


  • @pixel
    Sorry I cant help you to improve detection, but I’d like to say thank you for summarizing all the relevant installation steps - I was looking for this quite a while …

    BR,
    Thomas