Filter Rules disappearing

fbartels

no no luck reproducing it on our side and therefore also no fix for this. We have added two changes though to get some additional logging:

https://stash.kopano.io/projects/KC/repos/kopanocore/commits/5f7b0f351f78266a773d323f657b89ce34f60a61
https://stash.kopano.io/projects/KC/repos/kopanocore/commits/614469db65191788ef15d6416e7a83192a4afbf0

bmaehr

@fbartels I just have to add enough rules on my running instance and the rules get lost without any upgrade. I can provide you the information to reproduce it.

lherrmann

I tried to reproduce this with 66 kB of rules, unfortunately I wasn’t able to reproduce. Could you perhaps send us your rules? If you don’t want to post it here on the forum, you could alternatively send it to: feedback@kopano.io

To backup your rules:

kopano-backup --only-meta -u <username> -f Inbox

freyuh

This post is deleted!

freyuh

This post is deleted!

freyuh

Hello,

I have the same issue with the filter rules.

I have created about 20 rules in the WebApp. And after the next mail ist processed by dagent the rules disappeared in the WebApp.
But they can be displayed via kopano-rules.py --list. So they are still available.
And after they disappeared in the WebApp they don’t show up in the dagent log, so they aren’t used any more.
After I delete the last rule they show up again in the WebApp and in the dagent log. But they aren’t used by dagent. Every log entry says: Rule “***” does not match: not found (8004010f)
And afterwards the rules tab is empty again in the WebApp.

fbartels

@freyuh to prevent spam bots (usually newly registered accounts) from posting messages there is a moderation queue for new users which you tripped into three times. I have approved all three of your posts, maybe you want to trim them down into a single post (the largely cover the same, but each one has different details).

also posting the exact version where you encounter an issue would be helpful as well.

freyuh

Hello Felix,

sorry, the message disappeared out of the quick reply box and nothing happened so I thought, that I did something wrong …

Version info:
WebApp: 3.4.17.1613+72.1
Kopano Core: 8.6.80
Z-Push: 2.4.3+0-0

bmaehr

@lherrmann Is the problem fixed? Should I try with a new version?

compsos

Hi
We have seen the same issue of rules and signatures missing from accounts in 8.6.80 in both Debian Stretch and Jessie.
Will run those tests to see what we can find.

bmaehr

I’m really disappointed.

I found out what’s the problem, sent you the data for the testcase and 2 months later it is still not fixed…

Coffee_is_life

@bmaehr said in Filter Rules disappearing:

I’m really disappointed.

I found out what’s the problem, sent you the data for the testcase and 2 months later it is still not fixed…

greate you found the problem, since you did, would you share the solution?

best regards
coffee_is_life

jengelh

I have created about 20 rules in the WebApp. And after the next mail ist processed by dagent the rules disappeared in the WebApp. But they can be displayed via kopano-rules.py --list. So they are still available.

Well if kopano-rules.py shows any part of it and WebApp does not, that would initially point to WebApp.

And after they disappeared in the WebApp they don’t show up in the dagent log, so they aren’t used any more.
After I delete the last rule

If they are not shown in WebApp, there is nothing to delete.

they show up again in the WebApp and in the dagent log. But they aren’t used by dagent. Every log entry says: Rule “***” does not match: not found (8004010f)

If it is what I think it is, then: This message means that destination folder of the rule action was not found.

freyuh

Hello,

now I can’t even print the rules with the kopano-rules.py script.
I get the following error message:

Traceback (most recent call last):
File “/usr/local/bin/kopano-rules.py”, line 1110, in <module>
main()
File “/usr/local/bin/kopano-rules.py”, line 1104, in main
kopano_rule()
File “/usr/local/bin/kopano-rules.py”, line 885, in kopano_rule
printrules(filters, user, server)
File “/usr/local/bin/kopano-rules.py”, line 665, in printrules
condition_message = convertcondition(rule[4].Value)
File “/usr/local/bin/kopano-rules.py”, line 311, in convertcondition
totalconditions = len(conditions.lpRes)
TypeError: object of type ‘SPropertyRestriction’ has no len()

Ulrich.

jengelh

There is a new program, you can use kopano-ibrule -S (see manpage for the -u option too) instead.

freyuh

Hello,

hmm, kopano-rules was easier to read / use …

Ulrich.

robing

@freyuh ,

We rewrote kopano-rules so it can handle imports from exchange and listing rules was not a high priority at the moment.
We are still busy with it and are working on a fix for listing the rules at the moment .

Robin

freyuh

And once again, after adding several rules, all of them disappeared …
The WebApp shows no rules and kopano-ibrule -S shows nothing.

WebApp: 3.4.23.1909+83.1
Kopano Core: 8.7.80
Z-Push: 2.4.4+0-0

Ulrich.

ashceryth

@fbartels @robing

Hi guys,

This is still an issue and definitely a massive showstopper. Today, I completely reinstalled my setup on a new machine and migrated the database to the new host. After the db import I checked the WebApp and all rules were gone. To test with a “clean” db I deleted all rules for my user in WebApp on my old setup and exported the database again. So there were no rules in the db dump. After reimporting the db on the new host I recreated all rules (~25) manually via WebApp. The rules didn’t disappear while adding them. I also checked via kopano-ibrule command while adding the rules. Everything seemed to be fine. However, 1 hour later as some mails arrived that would be matched by one of the rules all of them are gone again! They are not visible via WebApp nor via kopano-ibrule command anymore.

@lherrmann, for me the behaviour is a 100% reproduceable.

I can’t send you my rules backup beacuse kopano-backup --only-meta -u <username> -f Inbox does not show any output in the rules file after the rules dissapeared. Trying to export the rules on the old host (using an db version where rules still existed) throws the following error:

kopano-backup --only-meta -u <username> -f Inbox
[error  ] gsoap connect: connect failed in tcp_connect()
2018-12-02 19:53:55,598 - backup0 - ERROR - Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/kopano/log.py", line 89, in log_exc
    try: yield
  File "/usr/lib/python2.7/dist-packages/kopano_backup/__init__.py", line 878, in dump_rules
    path = store.folder(entryid=_hex(f.text.decode('base64'))).path
  File "/usr/lib/python2.7/dist-packages/kopano/folder.py", line 240, in path
    parent = parent.parent
  File "/usr/lib/python2.7/dist-packages/kopano/folder.py", line 173, in parent
    parent_eid = _benc(self._get_fast(PR_PARENT_ENTRYID))
  File "/usr/lib/python2.7/dist-packages/kopano/compat.py", line 101, in benc
    return s.encode(_BIN_ENCODING).strip().upper()
AttributeError: 'NoneType' object has no attribute 'encode'

2018-12-02 19:53:55,598 - backup0 - ERROR - Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/kopano/log.py", line 89, in log_exc
    try: yield
  File "/usr/lib/python2.7/dist-packages/kopano_backup/__init__.py", line 878, in dump_rules
    path = store.folder(entryid=_hex(f.text.decode('base64'))).path
  File "/usr/lib/python2.7/dist-packages/kopano/folder.py", line 240, in path
    parent = parent.parent
  File "/usr/lib/python2.7/dist-packages/kopano/folder.py", line 173, in parent
    parent_eid = _benc(self._get_fast(PR_PARENT_ENTRYID))
  File "/usr/lib/python2.7/dist-packages/kopano/compat.py", line 101, in benc
    return s.encode(_BIN_ENCODING).strip().upper()
AttributeError: 'NoneType' object has no attribute 'encode'

When using python kopano-rules.py --user <username> --list the rules get listed but some of them have an empty condition. Those with the empty condition are filtering by the destination address. So maybe that causes the issue?

Please, let me provide anything else you need for digging into that. Just tell me what you need, please.

Debian 9.6
KC: 8.7.80.169-0+44.1
KW: 3.4.25.1980+1095.1
MariaDB Server: 10.1.37-0+deb9u1

Regards,
Patrick

ashceryth

I recreated 4 rules with conditions on sender address and content of subject only. Those seem to work, as they get applied and don’t disappear. So it seems it has to do with the condition of the rule…