Kopano setup recommendation

mlued

Greetings dear Kopano community,

we want to change our mailserver from MS Exchange to Kopano.
But before we make the big move, my boss wants a test system with a small group of users (early adopters (20-30).
To be future proof, we wanted a high availability system. In the documentation
Kopano recommends DRDB and pacemake. We had lots of trouble with DRDB in the past, so we set up a mariadb galera cluster (3 VMs).
Kopano is running also on these servers (Debian 10 with the shipped 8.7.0 Kopano Version, I know not ideal). The corresponding IPs for mariadb and kopano are handled by keepalived
Yesterday I tried to import a few mailboxes (kopano-migration-imap). To see progress I logged in to my account and I saw mails popping up.
After a while (I tried this and that) I decided to clean the mailboxes and try the import again. I disconnected the store with

kopano-storeadm -D -n username

And wiped it from the database with

kopano-storeadm -R <GUID>

While the new imap import started, I got the following message:

Host2 failure: Error login on [localhost] with user [username] auth [LOGIN]: 2 BAD Internal error: OpenECSession failed

In the kopano-server log I got lots of duplicate errors:

KDatabase::I_Update() query failed: "Duplicate entry.....

I then tried to fix the database issues with:

kopano-dbadm k-1216

But the errors persisted. My ultima ratio was to overwrite the database with an older backup. After that everything went smooth.

My question is, is it possible that I will experience a problem like this again on an import like this?
I can not risk to loose a lot of mails, when I have to recover an older backup because of such database inconsitencies.
The second question is, is there any experience with this combination of galera and kopano here?
And before the question will be raised, as soon as the test will proof a good and stable mailserver, we will buy kopano licenses and support.

greetings from germany

Maik

fbartels

Hi @mlued,

@mlued said in Kopano setup recommendation:

While the new imap import started, I got the following message:

Could it be that these log messages are from a client that was still connected at the time the old store was removed and therefore is still trying to open the old store? Do these messages go away after stopping kopano-gateway and restarting it after all clients have run into their individual timeouts?

@mlued said in Kopano setup recommendation:

Kopano recommends DRDB and pacemake. We had lots of trouble with DRDB in the past, so we set up a mariadb galera cluster (3 VMs).

In the end there are many ways to reach high availability and the key points should be to have some form of replication both for stored files (which is where attachments and imap data are stored) as well as for the database (which is where the actual mailbox data minus the attachments are stored). If you have a form of replicated storage on your network then its fine to use this directly instead of working with replicated file systems. The usage of Galera is also fine, but comes with the downside that you need at least three systems and each of them needs to hold a replica of all databases. Here it would be easier to just use MariaDB/MySQL Master/Master replication directly.

@mlued said in Kopano setup recommendation:

And before the question will be raised, as soon as the test will proof a good and stable mailserver, we will buy kopano licenses and support.

Its worthwhile though to restart your tests with an evaluation key and the actual supported packages. The version provided by Debian is gravely out of date and uses a different packaging layout compared to our own packages. So it will not be a case of simply swapping packages when doing the step towards an officially supported installation.

mlued

@fbartels
A long time passed, but now I am testing a evaluation license from kopano.
I really had to update a few installation routines, but all in all it went ok.
I even got a solution to my topic. Als I wrote, I removed the store from the user and deleted the then orphaned store.
But I forgot to create a new store with

kopano-storeadm -Cn <user>

I have one thing, that concerns me the most in the migration process.
We have an Microsoft Exchange server, with > 400 user, but the migration is pretty straight forward. (as far as I could test it, with some mailaccounts)
The second server is a dovecot mailserver. I tried two ways to migrate. At first, I used the mbox2kopano python script. It took forever and it dumped all mails in the inbox (not in subfolders as it is on the old server).
The next thing is kopano-migration-imap. It works nice but the speed is abysmal. Currently I have a test running, witch has an ETA of today 7 pm. The throughput is around 2.7-3 messages/second. Is this normal? I don’t know, how I should explain this to the users, when we will migrate their mailboxes and they should not use it for a whole day.

Is there a better (faster) way for the migration?

By the way, I changed the database to a master/master configuration and followed the recommendations from the documentation.

fbartels

@mlued said in Kopano setup recommendation:

with > 400 user

For these amounts of users I would rather recommend to talk to our support directly. They can also check server performance for you.

mlued

@fbartels
The import from Exchange is not that fast, but my own 300 MB mailbox took less than 30 min.
My question is rather, can I speed up the

kopano-migration-imap

import?

fbartels

@mlued said in Kopano setup recommendation:

can I speed up the […] import?

Most probably. In the end this all comes down to Innodb write performance.

PS: kopano-migration-imap is in the end just a packaged up version of imapsync so all its documented strategies for partial/time based migration apply here as well.