Filter Rules disappearing

ashceryth

Hey guys,

I am currently running KC version 8.6.80.621-0+63.1 on Debian Stretch and tried to upgrade to latest nightly (8.6.80.1014-0+150.1). Unfortunately the upgrade let all my Filter Rules disappear in Web App and non of them was working anymore. After restoring my backup snapshot of the VM running Kopano Core all the Rules were back there again. So it appears that the rules stay in the database* (or are they stored elsewhere?) but aren’t shown and applied anymore. Any suggestions?

Thanks!

* DB is on a different host and hasn’t been restored

fbartels

@ashceryth said in Filter Rules disappearing:

or are they stored elsewhere?

with the exception of attachment (and user data if you are using ldap) everything is stored in the database.

I did not attempt to reproduce this error, anyone else experiencing it?

ashceryth

@fbartels said in Filter Rules disappearing:

with the exception of attachment (and user data if you are using ldap) everything is stored in the database.

Well, that’s what I expected.

Is there anything I can provide so that you can gain some further insight on the issue?

fbartels

You could try to setup a new installation/mailbox and see if you get the same results there as well.

Its not like we discard some data during upgrade, so I don’t think it would make sense to look in the server.log, maybe raise logging of the dagent and see if it logs something interesting when a rule should be applied.

ashceryth

I tried upgrading the core again with nightly from last night. The issue persists. I raised the log level to 6 in both dagent and server, configuration, but there is really nothing relevant in the log files. I can create new rules which are then shown in the Web App. However, all old rules (rules that were created before upgrading) are gone…

When creating a new mailbox - as suggested by you - there won’t be any rules as they are local to the user, or isn’t it like that?

fbartels

@ashceryth said in Filter Rules disappearing:

When creating a new mailbox - as suggested by you - there won’t be any rules as they are local to the user, or isn’t it like that?

Yes, but I meant like “check if it’s something related to your inbox, or does it happen to other mailboxes as well”. For this you of course need to create a rule in the inbox before the upgrade.

ashceryth

Well, that makes sense now :) I will try that later and report the results. Very strange issue.

ashceryth

Maybe also interesting: The new rule that has been created after the upgrade for testing purposes has overwritten all old rules. So after restoring the KC VM snapshot, there was only this one rule so I had to restore the database as well.

umgfoin

@fbartels said in Filter Rules disappearing:

[…], anyone else experiencing it?

I observed a similar thing:
Webapp-settings got corrupted, entirely with one of the recent master-builds:

rules disappearing
inbox-privileges were broken (no access)
webapp started with initial setup-dialog

Re-creating the user fixed the problem for a short time, but corruption appeared again.

I didn’t report the issue, as I was too busy and not sure, if a build-problem was the main-cause on my system.
It happened with one of the recent master-builds (1- 2 weeks ago), but didn’t show up again, since.

++umgfoin.

lherrmann

I tried to reproduce it (updated from 8.6.2 to 8.6.80 in core), but my rules weren’t affected by the upgrade…

bmaehr

After upgrading to the latest nightly yesterday all my rules got lost.

Coffee_is_life

Hello @all

there is a script (https://stash.kopano.io/projects/KSC/repos/core-tools/browse/kopano-rules) which you can use to list all rules of a specific user. - Check if the rule is still shown.

coffee_is_life

bmaehr

@Coffee_is_life All gone.

Coffee_is_life

@bmaehr and i bet dagent log doesnt show lines like

[info   ] [32079] Rule <Some Rule_Name>  doesn't match: 0x8004010f

there must be somethink broken with the listing and processing of these rules, since they exist in DB.
even the python-script cant see or locate these rules

since you are using the nightly builds, i dont think you got subsription (if i’m wrong, best way would be to open a ticket)
this issue is huge and got a greate impact on daily workflow, so i think the devs will look into this asap - with and without ticket

coffee_is_life

bmaehr

dagent log is completely empty (created 3 months ago).
That’s the bad thing in kopano using the SQL db just as data heap: I can’t check if they are really there. I don’t think they will come back, but perhaps I get surprised.
The rule created with core-8.6.80.1018 didn’t get deleted after upgrading to core-8.6.80.1055, so it seams not to be a problem of the upgrade itself but more a problem of the format of the old rules.

Christoph.Goth

Same problem here, at least with one User. Another Users rules dont seem to be affected.

Problematical user recreated his rules, are visible with py-script but no more in webapp and dont get applyed :(

fbartels

Up until this point we were sadly not able to reproduce this issue. If you have any more information about the rules of the user, or with which master version this appeared first we would be happy to hear about this.

Christoph.Goth

I’ve upgraded from core-8.6.80.752_0+85-Debian_9.0-amd64 to core-8.6.80.1018_0+152-Debian_9.0-amd64, then the problem occured.
The User was greeted by the webapp configuration wizard, others not.
Afterwards, i upgraded to core-8.6.80.1055_0+156-Debian_9.0-amd64 with no change.

Then i recreated the rules in webapp, after switching windows in webapp they disappeared there again.

Using rules based on “Includes these words in the sender’s address” are showing up correctly in py-script, those using full senders address are having an empty Condition-Table entry. Maybe there the Problem comes from?

After Upgrade working users rules only use the first, working, condition.

Another thing i notices:
The problematic user is also missing the mail signature, setup via webapp.

bmaehr

@fbartels Can you provide a SQL Statement to list all rules from the database directly? So we could check if they are still there.

Christoph.Goth

After deleting the Rules with missing conditions, the remaining rules reappeeared in the Webapp, test if they’re working is pending.