kopano-spamd (8.6.2) is broken

weini

I´m facing the same problem with a current nightly build (core-8.6.80.1055_0+156-Debian_9.0-amd64). I definitely don´t use python 3.x but python 2.7 instead.

Moving an email to junk leads to the following log:

Jun 27 17:06:58 nas2 kopano-spamd[4004]: 2018-06-27 17:06:58,704 - spamd - ERROR - could not process change for entryid 0000000056286673185E4F68AD47C3C76D4A4F9A01000000
05000000357950C9DE80412A930F0A0D9C8FE41400000000 ([SPropValue(0x67110003, 1125145L), SPropValue(0x67150003, 1004340L), SPropValue(0x0FFA0102, 'V(fs\x18^Oh\xadG\xc3\xc7m
JO\x9a')]):
Jun 27 17:06:58 nas2 kopano-spamd[4004]: 2018-06-27 17:06:58,729 - spamd - ERROR - Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/kopano/ics.py", line 118, in ImportMessageChange
    self.importer.update(item, flags)
  File "/usr/lib/python2.7/dist-packages/kopano_spamd/__init__.py", line 81, in update
    elif item.folder == item.store.inbox and \
  File "/usr/lib/python2.7/dist-packages/kopano/store.py", line 194, in inbox
    return _folder.Folder(self, _benc(self.mapiobj.GetReceiveFolder(u'IPM', MAPI_UNICODE)[0]))
  File "/usr/lib/python2.7/dist-packages/MAPICore.py", line 611, in GetReceiveFolder
    return _MAPICore.IMsgStore_GetReceiveFolder(self, lpszMessageClass, ulFlags)
MAPIErrorNoSupport: MAPI error 80040102 (MAPI_E_NO_SUPPORT)

cburghardt

I don’t really get that logic. Kopano supports python3 if I am not mistaken. I am using packages for Suse Leap and they are only provided for python3. I get the same issue as above but already since 8.6.0.
Is there a workaround?

cburghardt

So what I did is to edit kopano_spamd/init.py and to convert the searchkey to binary:
self.learnham and self.was_spam(b’searchkey’)
self.mark_spam(b’searchkey’)

weini

Thanks for the hint!
I confirm, most of the error messages are gone, but not all:

Jul 26 08:22:36 nas2 kopano-spamd[25246]: 2018-07-26 08:22:36,940 - spamd - ERROR - could not process change for entryid 000000008D071BEA942F42619CEB29F47F6CC4CA010000000500000053F76E18C6A243A4A81AF24F0DB20A6900000000 ([SPropValue(0x67110003, 1130852L), SPropValue(0x67150003, 201L), SPropValue(0x0FFA0102, ‘\x8d\x07\x1b\xea\x94/Ba\x9c\xeb)\xf4\x7fl\xc4\xca’)]):
Jul 26 08:22:36 nas2 kopano-spamd[25246]: 2018-07-26 08:22:36,991 - spamd - ERROR - Traceback (most recent call last):
File “/usr/lib/python2.7/dist-packages/kopano/ics.py”, line 118, in ImportMessageChange
self.importer.update(item, flags)
File “/usr/lib/python2.7/dist-packages/kopano_spamd/init.py”, line 72, in update
item.header(headertag).upper() != ‘YES’:
NameError: global name ‘headertag’ is not defined

However, I don´t get anything placed into “/var/lib/kopano/spamd/[ham|spam]”

fbartels

@weini said in kopano-spamd (8.6.2) is broken:

NameError: global name ‘headertag’ is not defined

this has been fixed for a month already (in https://stash.kopano.io/projects/KC/repos/kopanocore/commits/dc15ffc9334b228b54539796e175c9cf7fd24f46) you should update your version.

weini

Thanks for the hint! Upgraded to build 1248 and added the two tweaks from cburghardt mentioned above.
Now I´m getting:

Jul 28 08:42:54 nas2 kopano-spamd[53301]: 2018-07-28 08:42:54,200 - spamd - ERROR - could not process change for entryid 0000000056286673185E4F68AD47C3C76D4A4F9A01000000050000000211D3BEB2E64C178164EE317BA80F8F00000000 ([SPropValue(0x67110003, 1131213L), SPropValue(0x67150003, 121928L), SPropValue(0x0FFA0102, ‘V(fs\x18^Oh\xadG\xc3\xc7mJO\x9a’)]):
Jul 28 08:42:54 nas2 kopano-spamd[53301]: 2018-07-28 08:42:54,200 - spamd - ERROR - Traceback (most recent call last):
File “/usr/lib/python2.7/dist-packages/kopano/ics.py”, line 118, in ImportMessageChange
self.importer.update(item, flags)
File “/usr/lib/python2.7/dist-packages/kopano_spamd/init.py”, line 81, in update
elif item.folder == item.store.inbox and
File “/usr/lib/python2.7/dist-packages/kopano/store.py”, line 194, in inbox
return _folder.Folder(self, _benc(self.mapiobj.GetReceiveFolder(u’IPM’, MAPI_UNICODE)[0]))
File “/usr/lib/python2.7/dist-packages/MAPICore.py”, line 611, in GetReceiveFolder
return _MAPICore.IMsgStore_GetReceiveFolder(self, lpszMessageClass, ulFlags)
MAPIErrorNoSupport: MAPI error 80040102 (MAPI_E_NO_SUPPORT)
Jul 28 09:36:17 nas2 kopano-spamd[53301]: 2018-07-28 09:36:17,180 - spamd - ERROR - could not process change for entryid 0000000056286673185E4F68AD47C3C76D4A4F9A01000000050000001E97A8A533C14154A28E4F33C1C8BDE700000000 ([SPropValue(0x67110003, 1131221L), SPropValue(0x67150003, 121928L), SPropValue(0x0FFA0102, ‘V(fs\x18^Oh\xadG\xc3\xc7mJO\x9a’)]):
Jul 28 09:36:17 nas2 kopano-spamd[53301]: 2018-07-28 09:36:17,181 - spamd - ERROR - Traceback (most recent call last):
File “/usr/lib/python2.7/dist-packages/kopano/ics.py”, line 118, in ImportMessageChange
self.importer.update(item, flags)
File “/usr/lib/python2.7/dist-packages/kopano_spamd/init.py”, line 81, in update
elif item.folder == item.store.inbox and
File “/usr/lib/python2.7/dist-packages/kopano/store.py”, line 194, in inbox
return _folder.Folder(self, _benc(self.mapiobj.GetReceiveFolder(u’IPM’, MAPI_UNICODE)[0]))
File “/usr/lib/python2.7/dist-packages/MAPICore.py”, line 611, in GetReceiveFolder
return _MAPICore.IMsgStore_GetReceiveFolder(self, lpszMessageClass, ulFlags)
MAPIErrorNoSupport: MAPI error 80040102 (MAPI_E_NO_SUPPORT)

So still no luck, it´s getting frustrating.
Is there some wiki page or doc how to track such issues down?

Spiff

Hi,

Upgraded to latest 8.6 (nightly) today, and have a similar problem.
In my case it seems like the item.header(self.headertag) fails into a NoneType object that does not support upper().
Get the following error whenever I move a message into the Junk folder:

2018-07-28 16:55:27,197 - spamd - ERROR - could not process change for entryid 000000003872FA2265094788A871E21E74BB150101000000050000004FE6C1B1D13743A0ABFB9AB828B4FCC100000000 ([SPropValue(0x67110003, 148806L), SPropValue(0x67150003, 134L), SPropValue(0x0FFA0102, ‘8r\xfa"e\tG\x88\xa8q\xe2\x1et\xbb\x15\x01’)]):
2018-07-28 16:55:27,197 - spamd - ERROR - Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/kopano/ics.py”, line 118, in ImportMessageChange
self.importer.update(item, flags)
File “/usr/lib/python2.7/site-packages/kopano_spamd/init.py”, line 69, in update
item.header(self.headertag).upper() != ‘YES’:
AttributeError: ‘NoneType’ object has no attribute ‘upper’

Update:
Removed “.upper()” from the script for testing, and got past this problem.
Obviously when the header does NOT contain “X-Spam-Flag”, an empty object is returned, hence it breaks the code here.
-Please update code to avoid this issue (e.g. test validity and default to “NO” if property is not found)

Second observation, amavis seems to almost be a requirement. -I’m using Spamassassin with clamav. -What should i set the sa-group to when amavis doesn’t exist?

mark dufour

hi all,

sorry for the problems you are having. I will take a look shortly at the mentioned issues.

the header problem, should be easy to solve
running spamd with python3, as now happens in master
the GetReceiveFolder issue

I wouldn’t use b’searchkey’, as this results in the literal word ‘searchkey’…

mark dufour

hi all,

quick update: I made a PR which should address above issues. it should land in master within a few days.

we now treat a missing header as signal the mail should be seen as spam
made some python3 fixes, and made sure our tests test spamd with python2 and python3
made store.inbox return None for public stores, avoiding the GetReceiveFolder issue

will post an update once this PR makes it through QA.

mark.

weini

Hi Mark!
It´s really great to see progress here, after this was kind a dead end for some months.
Many thanks for taking care.

Small comment on the default handling if the spam tag is missing: Best option would be to have this configurable via the cfg file. While your proposal is obviously the best approach to be on the safe side, there are several spam solution besides amavis that only add the tag on spam emails and leave ham emails untouched.

Chris

mark dufour

let’s first check if everything is better now. the patches have been merged to our 8.7 branch. I will let you know once they hit master.