Wrong indication of read/unread messages

genesis74

thanks for your quick answer. I now did the following:

log out of WebApp
clear cache with kopano-srvadmin --clear-cache all
restart kopano-server
log in to WebApp

Indication of unread counts at folder names were now in sync with number of messages found marked as unread in the respective folders. But reading those messages did not decrease the count at the folder name. For that, I had to chose “Mark all messages as read” at that folder. After that, logging out of and back in to WebApp did not change anything.

After restarting kopano-server those few messages in question are shown as unread again (some of them, but not all, in the unread email widget; no counters shown near folder; only messages in the folder shown as unread).

Maybe this can help finding the problem and its solution.

genesis74

@fbartels: Just noticed that now there is something common between the affected messages. They are all sent by my Fritz!Box Router (status messages). At the time of my first post there were additional messages affected, now (after the steps I described in my last post) only those Fritz!Box messages are left.

fbartels

Hi @genesis74,

are these messages particularly large? Could you provide us with an eml export of such a message? (you could for example send it to our support with a mention of KC-1417).

Thanks

genesis74

Hi @fbartels,

those messages are rather small and are consisting of html content only. No plain text part. I will check if I can send one to your support.

genesis74

@fbartels: Since in KC-1417 it is mentioned that the problem might be cache-related I checked the cache settings of my kopano server. For whatever reason, the cache_cell_size setting was commented out. I set the cache_cell_size to 2048M. According to the manual this seems to be a rather high setting in my case, and that new limit obviously has not been reached by far yet (looking at system memory usage). This seems to have helped - no more read/unread problems so far (at least I am unable to reproduce them with this setting).

afischer

Hi,
I think we suffer from the same issue.
Since we updated to Kopano 8.7 last week, users report problems with read/unread messages. Mostly, Outlook 2010 with mapi-client were affected.
Besides swap-usage grows rapidly, so that we had a crash of kopano-server last week.
I couldn’t find a pattern so far. Moving all unread messages to a subfolder, marking all remaining elements read and moving back the unread fixed it.
Best regards,
Achim

genesis74

After running kopano-server for a few days without interfering (i.e. restarting) I had to do a reboot today. The described problem came up again. Now with emails not having anything obvious in common. Also, IMAP is now affected as well.

fbartels

Hi @genesis74 and @afischer,

while we are still not able to reproduce the effect we have yesterday merged a potential fix for this. I have uploaded a bug containing this fix (along with some other changes reported in 8.7.0) to https://download.kopano.io/supported/core:/pre-final/.

Can you give this a try and report back if the issue was resolved afterwards?

afischer

Hello Felix,
I’m unsure how to install the update. Shall I copy my kopano-repo-file and change the url from …final… to …pre-final… and then update the system with this repo enabled?
Is it sufficient to use kopano-core or do I need any other repo - e.g. kopano-webapp?
We are running z-push on a different machine, shall I install the updates there, too?
Best regards,
Achim

fbartels

Hi @afischer,

yes you could do it exactly like that. just add another repo file, but this one pointing towards pre-final instead of final. WebApp does not need to be updated for this, your z-push system could most probably also stay on this version.

BUT: we know meanwhile that the issue still occurs with this build. Still trying to nail down exact repro steps.

afischer

This Bug is very annoying. When can we expect a fix for this?
Is it possible to downgrade to the latest working version (8.6.9)?

Best regards,
Achim

genesis74

Indeed the problem still persists. The only way I can get rid of those unread flags is:

kopano-admin --reset-folder-count <user>
kopano-srvadm --clear-cache all
mark all messages as read with IMAP client
restart kopano-server

It took several iterations of the above mentioned procedure with decreasing numbers of unread-flagged-messages before I managed to have all messages flagged as read via IMAP as well as WebApp. I found no way to manage this without an IMAP client.

fbartels

Hi @afischer,

I have just now uploaded the same build that we run internally to https://download.kopano.io/supported/core:/pre-final/
This has a “fix” for emails being truncated after searching for them. the downside of this change is that even though these emails are not truncated any longer, they will only load as plain text.
at least on my test system I was still able to observe the mails being shown as unread and up until know I am still missing information to reproduce the missing webapp settings, so this is probably still in there as well.

@genesis74 you steps seem a bit overkill to me. What I could observe (since I have my WebApp usually instructed to not automatically mark mails as read) is that when WebApp prefetches the message or when I click on the message it turns to read. meaning that the message are actually stored as read on the database only that webapp does not realise it at first.

afischer

Hello Felix,
Hm, we did not notice the problem you fixed. (truncated emails)
Our users, especially my boss, get impatient due to the bug of unread messages. I don’t want to install a beta-version with potentially new bugs, in which this annoying bug still exist.
But you don’t answer my question if there is a way to downgrade to 8.6.9
Best regards,
Achim

fbartels

@afischer said in Wrong indication of read/unread messages:

But you don’t answer my question if there is a way to downgrade to 8.6.9

I have not tested this myself, but our support has some experience in this area. I’d recommend to talk to then.

genesis74

Hi @fbartels,

I am not sure about your last sentence. Because it seems to me that this is the point - message state “read” is not stored in the database because after logging out of WebApp and logging back in (see my first post) or after restarting kopano-server messages that had already been flagged as read are marked as unread again.

By using IMAP as a second way to accessing the database I experienced that some of those messages appear as unread also in my IMAP client after doing the kopano-admin --reset-folder-count step. It made me believe that the “read”-state might have been written to the cache but not to the database (assuming WebApp and kopano-gateway/IMAP are not using the same cache or at least not in the same way).

genesis74

Hi @fbartels ,

my observation is as described in my first post. Especially after restarting kopano-server the unread-flagged messages turn up again. But sometimes also after logging out of WebApp and logging back in. The number of unread messages in the folder is then often inconsistent with the number shown next to the folder name. And all this is inconsistent with what I see if I use an IMAP-client. The only way I found to get this into line is what I described in my last post - yes, it looks like overkill, but it was the only solution I could find.

By the way - reading your last sentence I am a little bit in doubt if the database is updated correctly when marking those emails as read. In that case, clearing the cache should solve the problem completely, shouldn’t it? It seems to me as if the problem is that sometimes the new state of the read/unread flag is only saved to the cache but not the database …

fbartels

@genesis74 said in Wrong indication of read/unread messages:

I am a little bit in doubt if the database is updated correctly when marking those emails as read

yes, that is the current theory. we have multiple tables with similar data (e.g. properties, mvproperties, tproperties) there seems to me a mismatch there.

8GM-DK

Adding my voice here to confirm that the issue is also affecting us as well as (at least) one of our customers. We tried updating to the latest (ubuntu) final (core, webapp, z-push…) but it did not help.

8GM-DK

FYI: We could fix the issue with

kopano-admin --reset-folder-count $USER

It seems the cache is confused, and just clearing it did not help. This command updated the folders correctly.