[Solved] Migrate productive ZCP to fresh clean Univention Kopano

Horst_Wassenberg

Hi there,

I have an issue which I assume to be not unique to me.

I have a ZCP production server 7.2.1 incl. z-push working.
I installed a new and clean Univention Kopano 8.3.1.32.
Both are VMs on Xen working well without problems.

I want to migrate content from ZCP to Kopano.

I saw how to uninstall ZCP and install Kopano and migrate on one system. This process seems heavy and outcome is my old system with all inherits (wanted and unwanted) of the past. This dislike and want to avoid.

I want to make my new Univention system have all mails, contacts and calendar entries.

I found imapsync to transfer mails. I made it working.
I could copy & pase contacts. Not nice but works.

I did not find a way to migrate the calendar. This is my remaining issue. I did not find a way to migrate all content of the exchange active sync process.

I tried a mysqldump from ZCP to import to Kopano. Failed. No solution found.

I would prefer to learn how to import a mysqldump from ZCP into Kopano. Or see a tool that hooks on both exchange active servers and moves all content like imapsync does for mails only.

Please help !

Regards,
Horst

fbartels

@horst_wassenberg said in Migrate productive ZCP to fresh clean Univention Kopano:

I tried a mysqldump from ZCP to import to Kopano. Failed. No solution found.
I would prefer to learn how to import a mysqldump from ZCP into Kopano

The first question to such statements is always: what have you tried and which error messages did you get?

Instead of doing imapsync you could also use kopano-backup to move inboxes from your old server to the new one, but here I would also recommend to upgrade to a more recent release 8.3 is already quite old. (The Univention app description has instructions on updating).

Horst_Wassenberg

@fbartels

Hi Felix, thanks for your ideas.

I dumped database zarafa and wrote it to database kopano on new system. No errors but not content arrived. I could read in internet that others also did not mange to transfer data this way. Should it work in theory or not?

I tried kopano-back. Last time, could not connect. I thought it may be linked to my free community licence. Now I tried again and could make it start to work. I used -c 192.168.xx.yy:236. So I was lacking http:// which was the issue. I interruped the transfer. There were some mapi-warning messages but the script seemed to proceed. I saved the command line I used.

kopano-backup -c http://192.168.xx.yy:236 -U user -P pass

I updated all packages. Then tried kopano-backup again. I used the stored command line as I am a lazy person. Now get below error from kopano-backup.

So I am lost again … sorry. Still I would like to understand whether a sql dump from zarafa and moving it into the kopano-database should work or not. I have 8 GB mailbox so kopano-backup would need hours anyhow I think. Dumping, FTPing and loading more efficient …

Regards,
Horst

2018-05-24 21:49:49,871 - backup - ERROR - Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/kopano/log.py", line 87, in log_exc
    try: yield
  File "/usr/lib/python2.7/dist-packages/kopano/service.py", line 180, in start
    _daemon_helper(self.main, self, self.log)
  File "/usr/lib/python2.7/dist-packages/kopano/service.py", line 59, in _daemon_helper
    func()
  File "/usr/lib/python2.7/dist-packages/kopano_backup/__init__.py", line 349, in main
    self.backup()
  File "/usr/lib/python2.7/dist-packages/kopano_backup/__init__.py", line 360, in backup
    jobs = self.create_jobs()
  File "/usr/lib/python2.7/dist-packages/kopano_backup/__init__.py", line 563, in create_jobs
    (not servers or public_store.home_server in servers):
  File "/usr/lib/python2.7/dist-packages/kopano/store.py", line 627, in home_server
    for node in self.server.nodes(): # XXX faster?
  File "/usr/lib/python2.7/dist-packages/kopano/server.py", line 216, in nodes
    for row in self.table(PR_EC_STATSTABLE_SERVERS).dict_rows():
  File "/usr/lib/python2.7/dist-packages/kopano/server.py", line 220, in table
    return Table(self, self.mapistore.OpenProperty(name, IID_IMAPITable, MAPI_UNICODE, 0), name, restriction=restriction, order=order, columns=columns)
  File "/usr/lib/python2.7/dist-packages/MAPICore.py", line 291, in OpenProperty
    def OpenProperty(self, *args): return _MAPICore.IMAPIProp_OpenProperty(self, *args)
MAPIErrorNoAccess: MAPI error 80070005 (MAPI_E_NO_ACCESS)

fbartels

@horst_wassenberg said in Migrate productive ZCP to fresh clean Univention Kopano:

Should it work in theory or not?

of course. dumping and restoring a database is actually a rather simple task. The creating the backup is explained in https://documentation.kopano.io/kopanocore_administrator_manual/backup_restore.html#full-database-dump a restore is then as simple as https://stackoverflow.com/a/105798/4754613.

@horst_wassenberg said in Migrate productive ZCP to fresh clean Univention Kopano:

MAPIErrorNoAccess: MAPI error 80070005 (MAPI_E_NO_ACCESS)

that is a strange error. did you try to clean out the cancelled backup data? did you upgrade to a more recent version as I suggested?

Horst_Wassenberg

@fbartels

Dumping and restoring is in my crontab since years, so I am very familar. I was doubting that some table / labels in the DB may have changed from zarafa to kopano. Anyhow, this is what happened:

to transfer mysql dump from zarafa to fresh kopano one needs:

have the user created in kopano
have the kopano-server stop (or not) and restart
restore the mysql dump

I had the idea to change sequence. First restore database without user created in kopano and create user in kopano only after restore of mysql database. So I tried. It worked! Surprise.

Unfortunately, I did use an old dump. OK. Created new dump. Did about same. Did not work. Grumpf.

I tried systematically:

a)

create new user in kopano
stop kopano-server
restore mysql dump
start kopano-server
=> did not work

b)

stop kopano-server
restore mysql dump
create new user in kopano
start kopano-server
=> did not work

c)

do NOT stop kopano-server
restore mysql dump
create new user in kopano
start kopano-server
=> did not work

The text “did not work” means restore without error. After restore, front ends totally empty. Looks like nothing was restored. Mailbox empty. Contacts empty. Calender empty.

Then I had the idea that mysql database on old zarafa system may be corrupt. I tried very old dump. No success eigther. So I gave up.

I now use imapsync. Did copy & paste contacts. Will re-create calendar from scratch. This will give a clean system which also has some advantage.

I am sorry I will not be able to do further testing.

For the sake of other users, perhaps someone can add how it worked in other case. Especially the sequence above.

One more note: I had attachements stored in database, not files. I always changed server.conf file. After, deleted mysql database and created a new one. When using imapsync, that worked well. For my final set-up, I changed to have attachements stored in files.

Regards,
Horst

fbartels

Creating a dump while the server is running should be no problem (as long as you use --single-transaction), but for restoring you should really shut down kopano-server. The server does not check if caches need to be invalidated and therefore will not detect any database changes not done through it.

Creating users before will not make a dofference, because you should drop tables/databases before restoring. Merging two database sets will just give you an inconsistant/not working state.

Are you maybe using an LDAP backend? If unique user IDs have changed, existing stores will be orphaned. This could also explain, why you don’t see data after import.

Edit: since you are running Univention you will be running with an LDAP backend indeed. It seems you created new users on your new server instead of fully replicating your old user tree. You will find your user data in the orphaned stores.

Horst_Wassenberg

@fbartels

yes, I did use --singletransaction and I did drop the database and did create a new one on command line in mysql.

Yes, likely in orphaned stores. By the way … where are orphaned stores ;-) ?

Last not least this is imapsync which worked well:

imapsync --host1 home --user1 horst --passfile1 home --host2 kopano --user2 horst --passfile2 kopano --addheader --nofoldersizesatend --noabletosearch --nofoldersizes --nofoldersizes --releasecheck --exclude “^Public folders”

Dear Felix, if you could let me know “now” where to find orphants … I may look at it … sorry … and THANKS so much so far. I think some items of this post could also help others …

fbartels

You will find your answer in the admin manual.

Horst_Wassenberg

Hi there,

@fbartels and me finally found the issue.

Univention is using LDAP. The restore of the mysql dump creates an orphant. This is why the front ends were always empty.

This is the link about how to link an orphant to a user:

https://documentation.kopano.io/kopanocore_administrator_manual/user_management.html?highlight=orphan#reattaching-stores-from-deleted-users

This finding and action closes the topic. Thanks so much to @fbartels

Regards,
Horst