KC 8.7.80 upgrade issues

smhaller

Hi Felix

No - a downgrade is even more horrable than failing forward.
So as of now:
Sending mails works
Retrieving mails not

Mon May 13 09:59:19 2019: [notice ] Client [::ffff:153.46.253.213]:43409 thread exiting
Mon May 13 10:00:27 2019: [info   ] Accepted connection from [::ffff:153.46.253.213]:43413
Mon May 13 10:00:27 2019: [notice ] Starting worker process for IMAP request
Mon May 13 10:00:27 2019: [debug  ] > * OK [CAPABILITY IMAP4rev1 LITERAL+ STARTTLS AUTH=PLAIN] IMAP gateway ready
Mon May 13 10:00:27 2019: [debug  ] < kman1 CAPABILITY
Mon May 13 10:00:27 2019: [debug  ] > * CAPABILITY IMAP4rev1 LITERAL+ STARTTLS AUTH=PLAIN CHILDREN XAOL-OPTION NAMESPACE QUOTA IDLE
Mon May 13 10:00:27 2019: [debug  ] > kman1 OK CAPABILITY Completed
Mon May 13 10:00:27 2019: [debug  ] < kman2 STARTTLS
Mon May 13 10:00:27 2019: [debug  ] > kman2 OK Begin TLS negotiation now
Mon May 13 10:00:27 2019: [info   ] Using SSL now
Mon May 13 10:00:27 2019: [debug  ] < kman3 CAPABILITY
Mon May 13 10:00:27 2019: [debug  ] > * CAPABILITY IMAP4rev1 LITERAL+ AUTH=PLAIN CHILDREN XAOL-OPTION NAMESPACE QUOTA IDLE
Mon May 13 10:00:27 2019: [debug  ] > kman3 OK CAPABILITY Completed
Mon May 13 10:00:27 2019: [debug  ] < kman4 AUTHENTICATE PLAIN
Mon May 13 10:00:27 2019: [debug  ] > + 
Mon May 13 10:00:27 2019: [debug  ] Initializing provider "Kopano Directory Service"
Mon May 13 10:00:27 2019: [debug  ] Initializing provider "Private Folders"
Mon May 13 10:00:27 2019: [error  ] HrGetStore failed: No store present.
Mon May 13 10:00:27 2019: [warning] Failed to login from [[::ffff:153.46.253.213]:43413] with invalid username "shaller" or wrong password: not found (8004010f)
Mon May 13 10:00:27 2019: [debug  ] > kman4 BAD Internal error: OpenECSession failed
Mon May 13 10:00:27 2019: [debug  ] < kman5 LOGOUT
Mon May 13 10:00:27 2019: [debug  ] > * BYE server logging out
Mon May 13 10:00:27 2019: [debug  ] > kman5 OK LOGOUT completed
Mon May 13 10:00:27 2019: [notice ] Disconnecting client.

The server connection to LDAP is now working as following confirms.

kopano-admin -l
User list for Default(4):
	Username	Fullname	Homeserver	
	-------------------------------------------------
	SYSTEM		SYSTEM		Kopano	
	shaller		Stefan Haller		
	ehaller		Elena Haller		
	challer		Cornelia Haller

But it seems that Kopano doesn’t know about the user store

kopano-admin --details shaller
User object id:         00000000AC21A95040D3EE48B319FBA7533044250100000006000000150000005533526C5A6D4675494568686247786C63673D3D00000000
WARNING: Unable to get user store entry id. User possibly has no store.

I think to remember that I once had such an issue in the past as well. But don’t remember how to solve it.

fbartels

@smhaller said in KC 8.7.80 upgrade issues:

No - a downgrade is even more horrable than failing forward.

Then I’ll wish you good look on your journey.

mcostan

@smhaller

How about you just setup a Debian 9 Virtual Machine, setup a very basic Kopano installation out of the box, then connect to the database and see if that works?

If it does, you know your database works and the issue is just with the Debian 10 setup. And the answer there would be just to roll back to Debian 9.

if instead it doesn’t work, then I would go back to your last good database backup…

smhaller

@mcostan said in KC 8.7.80 upgrade issues:

@smhaller

How about you just setup a Debian 9 Virtual Machine, setup a very basic Kopano installation out of the box, then connect to the database and see if that works

If it does, you know your database works and the issue is just with the Debian 10 setup. And the answer there would be just to roll back to Debian 9.

if instead it doesn’t work, then I would go back to your last good database backup…

I could for sure do this, but the access to the database works, I don’t need to proove this.
The issue is somewhere in the MAPI -> python3 relation, at least to my understanding.
As reported in the thread:
https://forum.kopano.io/topic/2014/kopano-spamd-not-starting-because-of-no-module-named-_mapicore-no-module-named-_mapicore/2

I also see the same issue. He somehow downgraded the system. most probably by a fresh install, which is for me not the way to go.

As per Debian specification a downgrade is not forseen. Even if I put the repos back to stretch, which did meanwhile, It is not supportive in going back with the software installed.

mcostan

@smhaller

My advise on this is never to run on installations directly on physical servers.

I learned the hard way a few years back that although I run on physical hardware, I have nothing on it.

All it does is to run VirtualBox and everything runs in Virtual Machines.

Before I upgrade anything including Kopano, I take a physical copy of the virtual machine, perform the upgrade, if it works great, otherwise I just roll back to the previous version of the virtual machine.

This way I literally have nothing to do, roll back files whatever, I just copy the disk of the virtual machine back and I am done with the rollback.

I would strongly advise doing this in the future. At least this is the way I do it.

In addition once a week automatically the virtual machines are backed up, even without an upgrade performed.

fbartels

@smhaller said in KC 8.7.80 upgrade issues:

The issue is somewhere in the MAPI -> python3 relation, at least to my understanding.

But neither kopano-dagent nor kopano-gateway have a direct relation to python. while the dagent has this plugin framework to to processing of message before the final delivery, the delivery is not related to python at all.

mcostan

@smhaller

Also this:

But it seems that Kopano doesn’t know about the user store
kopano-admin --details shaller
User object id: 00000000AC21A95040D3EE48B319FBA7533044250100000006000000150000005533526C5A6D4675494568686247786C63673D3D00000000
WARNING: Unable to get user store entry id. User possibly has no store.

I don’t think it is related to python? It appears something is now wrong in the database or am I wrong?

thctlo

Well you have hit multple bugs.

Some can be fixed easy, some there are workarounds for.

run :
kopano-admin --create-store shaller-c shaller@domain.tld -p PassWher -e email@domain.tld
kopano-storeadm -h default: -Cn shaller

and when the password isnt working run : kopano-admin -P -u shaller

try that,

smhaller

Thanks mate
that was a good starting point.

@thctlo said in KC 8.7.80 upgrade issues:

run :
kopano-admin --create-store shaller-c shaller@domain.tld -p PassWher -e email@domain.tld
kopano-storeadm -h default: -Cn shaller

Meanwhile the username to store is hooked again.

kopano-admin -u shaller --hook-store 524848af999946e492b5fbdf4aa4afe1

Now I can see that even login timestamp is updated when checking with kopano-admin --detail shaller

But the server doesn’t really let me in. On my mobile I get reported that “NO LOGIN imap feature is disabled”. Which in fact matches with the settings given.

Mapped properties:
	PR_EC_ENABLED_FEATURES	mobile; outlook; webapp	
	PR_EC_DISABLED_FEATURES	imap; pop3

Why are my client settings not working anymore?

mcostan

IMAP is indeed disabled.

Try this:

kopano-admin -u username --enable-feature imap

smhaller

Hmm yes I thought about that. But the settings from kopano should be as they were before

kopano-admin -v -u shaller --enable-feature imap
Unable to update user information: action not supported by server (0x80040102)
Using the -v option (possibly multiple times) may give more hints.

where server.log tells me:

Mon May 13 12:30:15 2019: [warning] K-1505: Unable to set details for external user 21: Change object is not supported when using the LDAP user plugin.

Meanwhile also incoming email is correctly delivered to the user stores. The only residing issue now is: to retrieve the mails with client.
In fact Kopano Webapp is not yet working, as the mapi.so cannot be found.

May 13 12:39:02 oklahoma sessionclean[6769]: PHP Warning:  PHP Startup: mapi: Unable to initialize module
May 13 12:39:02 oklahoma sessionclean[6769]: Module compiled with module API=20151012
May 13 12:39:02 oklahoma sessionclean[6769]: PHP    compiled with module API=20170718
May 13 12:39:02 oklahoma sessionclean[6769]: These options need to match
May 13 12:39:02 oklahoma sessionclean[6769]:  in Unknown on line 0

thctlo

run dpkg -l | grep mapi
whats the output?

enable the kopano mapi.
phpenmod kopano

the stop and start your webserver, (no restart, stop/start) .

smhaller

@thctlo said in KC 8.7.80 upgrade issues:

dpkg -l | grep mapi

ii  libkcicalmapi0                            8.7.80.936.529d71bca-0+193.1                                       amd64        iCal interface for MAPI
ii  libkcicalmapi0-dbgsym                     8.7.80.936.529d71bca-0+193.1                                       amd64        Debug symbols for libkcicalmapi0
ii  libkcinetmapi0                            8.7.80.936.529d71bca-0+193.1                                       amd64        Interface between internet e-mail and MAPI
ii  libkcinetmapi0-dbgsym                     8.7.80.936.529d71bca-0+193.1                                       amd64        Debug symbols for libkcinetmapi0
ii  libmapi1                                  8.7.80.936.529d71bca-0+193.1                                       amd64        Kopano's implementation of the Messaging API
ii  libmapi1-dbgsym                           8.7.80.936.529d71bca-0+193.1                                       amd64        Debug symbols for libmapi1
rc  php-mapi                                  8.7.0-3                                                            amd64        Complete and feature rich groupware solution - PHP MAPI bindings
ii  php7-mapi                                 8.7.80.936.529d71bca-0+193.1                                       amd64        PHP bindings for MAPI
ii  php7-mapi-dbgsym                          8.7.80.936.529d71bca-0+193.1                                       amd64        Debug symbols for php7-mapi
ii  python3-mapi                              8.7.80.936.529d71bca-0+193.1                                       amd64        Python 3 bindings for MAPI
ii  python3-mapi-dbgsym                       8.7.80.936.529d71bca-0+193.1                                       amd64        Debug symbols for python3-mapi

As you can see from this, the mapi packages are installed.

smhaller

Issue with php-mapi is now fixed.
I’m able to login through Kopano WebApp and read my email.

Remaining issue, login with outlook, mobile client not possible.

UPDATE: got connection from mobile phone and Mac Mail working.
Had to change the parameter in server.cfg
from: disabled_features = imap pop
to: disabled_features = pop

Honestly I have no clue why, because with 8.4 this setting was working. users in LDAP have all the setting enabledKopanoFeatures: imap

I also notified that gateway isn’t using MD5 as I thought it should be.

Cheers
Stefan

thctlo

@smhaller said in KC 8.7.80 upgrade issues:

rc php-mapi

remove that one php-mapi…

and apt-get remove *-dbgsym

smhaller

@thctlo said in KC 8.7.80 upgrade issues:

@smhaller said in KC 8.7.80 upgrade issues:

rc php-mapi

remove that one php-mapi…

ok - I thought it is still needed …

@thctlo said in KC 8.7.80 upgrade issues:

and apt-get remove *-dbgsym

yes that makes sense.

thctlo

remove that one php-mapi…

ok - I thought it is still needed …

@smhaller said in KC 8.7.80 upgrade issues:

@thctlo said in KC 8.7.80 upgrade issues:

@smhaller said in KC 8.7.80 upgrade issues:

rc php-mapi

remove that one php-mapi…

ok - I thought it is still needed …

You need php7-mapi, the php-mapi has status RC. which means its removed, but old config may exist.

smhaller

So, now all issues are solved.

the last bit was related to Z-PUSH and the apache config. As written in the thread here :
https://forum.kopano.io/topic/1634/centos7-no-matter-what-i-get-access-denied-please-send-authorisation-information/5

I also had to add the line: SetEnvIf Authorization “(.*)” HTTP_AUTHORIZATION=$1
to my config.

Thanks to: @thctlo, @mcostan and @fbartels for your support and guidance much appreciated.