IMAP issue on Thunderbird with core-8.3.0~788_104.1

robgnu

Hello,

I’ve found a new issue in the IMAP functionality of Kopano Core when using Thunderbird.

First, the affected Version is core-8.3.0~788_104.1. On core-8.3.0~694_76.1 everything seems to be ok.

The problem: When the user opens a Folder like Inbox, sometimes all emails are shown. Mostly there is only one email! When the user clicks on another folder and after that back to the first one, Thunderbird mostly shows all mails. But: Everytime Thunderbird seems to reload all Headers of the mails stored in the actual folder.

For now I’ve “dirty” downgraded to core-8.3.0~694_76.1 - everything is okay for now. But it took many hours to find out that there must be an error in Kopano. For debugging I tried to log the error, but there where no useful information in debug-logfile. Just a big amount of email headers.

I appreciate your help and information if the problem is confirmed and/or solved.

Thank you

fbartels

Hi @robgnu ,

thanks for your report. I’ve created https://jira.kopano.io/browse/KC-559 to investigate it a bit further.

bosim

@robgnu said in IMAP issue on Thunderbird with core-8.3.0~788_104.1:

The problem: When the user opens a Folder like Inbox, sometimes all emails are shown. Mostly there is only one email! When the user clicks on another folder and after that back to the first one, Thunderbird mostly shows all mails. But: Everytime Thunderbird seems to reload all Headers of the mails stored in the actual folder.

This sounds wierd. If you have the possibility could you set the gateway in debug log mode (log level 6) and post the relevant (ideally obfuscated) part of the log? Would help me a lot. In the meanwhile I will try later today testing with Thunderbird and update the status in the JIRA ticket.

Thanks for reporting this!

Bo

robgnu

Hello,
thank you for opening an issue. I did changed the debug-level this morning and you can find the log-snippet here:

https://www.gnuschke.de/imap.txt

I can provide these additional information:

The database was a Zarafa 7.2.4 before. I migrated several weeks ago with no problems.
Attachments are stored in the filesystem.
Thunderbird is the newest version.
The problem occours on all machines in the office.
Every Thunderbird has 4 IMAP connections (different users) to the Kopano-Gateway.
All Kopano-users have the same problem.
Some folders are quite big. The inbox or sent-folder counts more than 2000 mails, many of them have big attachments (1-30 MByte).

I hope the information is helpful. I couldn’t find any errors in the logfile, but I’m not an IMAP expert.

Best regards.

bosim

Hi

@robgnu said in IMAP issue on Thunderbird with core-8.3.0~788_104.1:

Hello,
thank you for opening an issue. I did changed the debug-level this morning and you can find the log-snippet here:

Thank you for the log. Everything looks sane to me. The only difference, from what I know, between the two versions is some IMAP optimizations I did. I will have them reverted, and investigate further.

Thanks for reporting this!

Bo

Hello,

same problem in my configuration. With every new IMAP reload in all folders on all machines with thunderbird (win10 pro) I can see

all mails, only one mail, all mails, only one mail …

I’ve also migrated from Zarafa 7.2.4, my folders contail only up to 150 mails.
Setting debug level to 6 doesn’t produce any relevant message in gateway.log!

When I connect Thunderbird to a non Kopano imap server, there’s no problem.
When I use eM Client with Kopano server, there’s also no problem.

As described by robgnu the problem appeared by updating kopano core - without change on thunderbird version (45.7.1).

I use only imaps on port 993.

I hope, this will help to solve this small problem in the GREAT KOPANO!!

Thanks and regards.

hajuq

alexl

Same issue here.

Upgraded from ZCP 7.2.4 to Core 8.3.0~718-85.1 and CentOS6_PHP56 to CentOS7_PHP56.

Now running Core 8.3.0~849-111.1 but the problem is still there.

Best regards,

Alex

Hello,

just in the moment updated with core-8.3.0~989_5.1-Debian_8.0-amd64 -> my thunderbird imap problem is solved!!

Thank you,

hajuq

robgnu

Hello,
I updated to the latest version a few minutes ago. At first view I can confirm that this issue seems to be solved. The real test starts tomorrow. :)
Thank you,
Rob

robgnu

Hello,
after some testing I can not confirm. Sorry, but the error is not fixed. Thunderbird is still reloading all headers when the user changes the folder.
In case of fast folder changing, the folders seems to be empty. After switching back to another and back to the first one, the headers will load again.

Installed version is: 8.3.0~999 on Debian 8.

Update: I installed core-8.4.0~0_10 but the problem still persists. In this situation Kopano is not usable.

Steps to reproduce:

Goto INBOX -> Mails are visible
Goto SENT -> Mails are visible (mostly)
Incoming new Mail.
Goto INBOX -> Only the new mail is visible, all other mails are “away”.
Go back to SENT and then again to INBOX: Thunderbird reloads all Headers and the new Mail isn’t visible.

Greetings.

fbartels

Sounds like your issue is then not related to the imap optimisations at all. do you have any error messages or even deadlocks in your kopano-server logging?

robgnu

I’ve checked the logfile and could’n find any deadlocks. Now I changed the logging level to debug.
There are no error-messages at all. Only messages like

Mon Mar  6 12:10:03 2017: [debug  ] Accepted incoming connection from file:///var/run/kopano/server.sock
Mon Mar  6 12:10:03 2017: [debug  ] Previous message logged 19 times
Mon Mar  6 12:10:03 2017: [debug  ] End of session (logoff) 14064714813765144326
Mon Mar  6 12:10:03 2017: [debug  ] Accepted incoming connection from file:///var/run/kopano/server.sock
Mon Mar  6 12:10:03 2017: [debug  ] End of session (logoff) 13620168352211014533
Mon Mar  6 12:10:04 2017: [debug  ] Accepted incoming connection from 127.0.0.1
Mon Mar  6 12:10:04 2017: [debug  ] Writing out logon/logoff time cache (1/1 entries) to DB
Mon Mar  6 12:10:04 2017: [debug  ] Accepted incoming connection from 127.0.0.1

What else can I do to help debugging?

Gerald

My Thunderbird is also affected, I tried multiple versions of Thunderbird since I initially thought using a beta version of Thunderbird is the problem. Clicking “Get Messages” changes what Thunderbird displays in the INBOX Folder. The Problem usually starts occuring when new mail arrived: I see a folder with 20-30 mails, and the second the new mail arrived it shows only the one new mail.

I installed “EM Client” as an alternative to Thunderbird and even though this client uses IMAP as well it does not seem to be affected.

bosim

Hi

I found the commit that caused the trouble. I have submitted revertion, let us hope it will make the nightly tomorrow.

If you want to revert it your self and compile the commit is: 611536a8fc36186b74804a367ce54fc420b86de4

Thanks for your patience!

Bo

robgnu

Ok, I can report, that I’ve downgraded to version core-8.3.0~694_76.1-Debian_8.0-amd64 and started the kopanao-server with

ExecStart=/usr/sbin/kopano-server -F --ignore-database-version-conflict

to force the downgrade. I confirm (again) on this version the IMAP is working without problems. The IMAP is broken in version 8-3-0-788 and later. But I can’t say, which version introduced the bug between 694 and 788.

bosim

Fix is now in kc-8.3.x, will ensure it will hit the master today, so it will get included in the nightly build.

Again, thanks for your patience.

Bo

Gerald

I’ve just installed the new build and my quick initial tests seem to confirm that tonights build has indeed fixed the IMAP problem.
Let’s hope it stays this way.

robgnu

Hello. Thank you for this information. I will check in a few days when I’m back home. I will report if the problem is solved.
Thanks to the Kopano-Team.

bosim

@Gerald said in IMAP issue on Thunderbird with core-8.3.0~788_104.1:

I’ve just installed the new build and my quick initial tests seem to confirm that tonights build has indeed fixed the IMAP problem.
Let’s hope it stays this way.

Glad to hear that!

Bo