Z-Push connection fails on IMAP and combined backend

[kitserve]

Hi there! We use Z-Push on Debian with the combined backend. It was manually installed from a tarball on Debian Jessie and worked fine. After upgrading our server to Debian Stretch it stopped working, so we tried switching to the packaged version from the official repositories. It still doesn’t work, and it doesn’t seem to produce any debugging information.

We set up the repositories as per https://wiki.z-hub.io/display/ZP/Installation. The packages currently installed are:
root@host:~# dpkg -l | grep z-push
ii z-push-backend-caldav 2.3.7+0 all Z-Push caldav backend
ii z-push-backend-carddav 2.3.7+0 all Z-Push carddav backend
ii z-push-backend-combined 2.3.7+0 all Z-Push combined backend
ii z-push-backend-imap 2.3.7+0 all Z-Push imap backend
ii z-push-common 2.3.7+0 all open source implementation of the ActiveSync protocol
ii z-push-ipc-sharedmemory 2.3.7+0 all Z-Push ipc shared memory provider

To rule out complications with the combined backend I have tested it with the IMAP, Caldav and Carddav backends each separately. In each case the result is the same: testing the ActiveSync URL in a browser works fine, but setting up an iOS device to access an account always results in this error:

Exchange Account
Unable to verify account information

The error log z-push-error.log is empty. Here are the entries in the main z-push.log from a test access from a browser:

05/09/2017 22:19:46 [25671] [DEBUG] [user] -------- Start
05/09/2017 22:19:46 [25671] [DEBUG] [user] cmd='' devType='' devId='' getUser='user' from='1.2.3.4' version='2.3.7+0' method='GET'
05/09/2017 22:19:46 [25671] [DEBUG] [user] Used timezone 'Europe/London'
05/09/2017 22:19:46 [25671] [DEBUG] [user] Including backend file: '/usr/share/z-push/backend/imap/imap.php'
05/09/2017 22:19:46 [25671] [DEBUG] [user] Including backend file: '/usr/share/z-push/backend/caldav/caldav.php'
05/09/2017 22:19:46 [25671] [DEBUG] [user] Including backend file: '/usr/share/z-push/backend/carddav/carddav.php'
05/09/2017 22:19:46 [25671] [DEBUG] [user] Combined 3 backends loaded.
05/09/2017 22:19:46 [25671] [DEBUG] [user] Request::ProcessHeaders() ASVersion: 14.0
05/09/2017 22:19:46 [25671] [DEBUG] [user] ZPush::CommandNeedsAuthentication(0): true
05/09/2017 22:19:46 [25671] [DEBUG] [user] Combined->Logon('user', '',***))
05/09/2017 22:19:46 [25671] [DEBUG] [user] BackendIMAP->Logon(): User 'user' is authenticated on '{example.com:993/imap/ssl/norsh}'
05/09/2017 22:19:47 [25671] [DEBUG] [user] BackendCalDAV->Logon(): User 'user' is authenticated on CalDAV 'https://example.com:443/owncloud/remote.php/caldav/calendars/user/'
05/09/2017 22:19:47 [25671] [DEBUG] [user] BackendCardDAV->Logon(): User 'user' is authenticated on 'https://example.com:443/owncloud/remote.php/carddav/addressbooks/user/'
05/09/2017 22:19:47 [25671] [DEBUG] [user] BackendCardDAV::discoverAddressbooks() Found addressbook 'https://example.com:443/owncloud/remote.php/carddav/addressbooks/user/contacts/'
05/09/2017 22:19:47 [25671] [DEBUG] [user] Combined->Logon() success
05/09/2017 22:19:47 [25671] [DEBUG] [user] NoPostRequestException: This is the Z-Push location and can only be accessed by Microsoft ActiveSync-capable devices - code: 2 - file: /usr/share/z-push/index.php:90
05/09/2017 22:19:47 [25671] [ INFO] [user] User-agent: 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0'
05/09/2017 22:19:47 [25671] [DEBUG] [user] ZPush::PrintZPushLegal()
05/09/2017 22:19:47 [25671] [ INFO] [user] cmd='' memory='1.70 MiB/2.00 MiB' time='0.50s' devType='' devId='' getUser='user' from='1.2.3.4' idle='0s' version='2.3.7+0' method='GET' httpcode='200'
05/09/2017 22:19:47 [25671] [DEBUG] [user] -------- End

And here are the entries from an attempt to activate an account on an iOS device:

05/09/2017 22:23:20 [27718] [DEBUG] [user] -------- Start
05/09/2017 22:23:20 [27718] [DEBUG] [user] cmd='' devType='' devId='' getUser='user' from='1.2.3.4' version='2.3.7+0' method='OPTIONS'
05/09/2017 22:23:20 [27718] [DEBUG] [user] Used timezone 'Europe/London'
05/09/2017 22:23:20 [27718] [DEBUG] [user] Including backend file: '/usr/share/z-push/backend/imap/imap.php'
05/09/2017 22:23:20 [27718] [DEBUG] [user] Including backend file: '/usr/share/z-push/backend/caldav/caldav.php'
05/09/2017 22:23:20 [27718] [DEBUG] [user] Including backend file: '/usr/share/z-push/backend/carddav/carddav.php'
05/09/2017 22:23:20 [27718] [DEBUG] [user] Combined 3 backends loaded.
05/09/2017 22:23:20 [27718] [DEBUG] [user] Request::ProcessHeaders() ASVersion: 14.0
05/09/2017 22:23:20 [27718] [DEBUG] [user] Combined->Logon('user', '',***))
05/09/2017 22:23:21 [27718] [DEBUG] [user] BackendIMAP->Logon(): User 'user' is authenticated on '{example.com:993/imap/ssl/norsh}'
05/09/2017 22:23:21 [27718] [DEBUG] [user] BackendCalDAV->Logon(): User 'user' is authenticated on CalDAV 'https://example.com:443/owncloud/remote.php/caldav/calendars/user/'
05/09/2017 22:23:21 [27718] [DEBUG] [user] BackendCardDAV->Logon(): User 'user' is authenticated on 'https://example.com:443/owncloud/remote.php/carddav/addressbooks/user/'
05/09/2017 22:23:21 [27718] [DEBUG] [user] BackendCardDAV::discoverAddressbooks() Found addressbook 'https://example.com:443/owncloud/remote.php/carddav/addressbooks/user/contacts/'
05/09/2017 22:23:21 [27718] [DEBUG] [user] Combined->Logon() success
05/09/2017 22:23:21 [27718] [DEBUG] [user] NoPostRequestException: Options request - code: 1 - file: /usr/share/z-push/index.php:66
05/09/2017 22:23:21 [27718] [DEBUG] [user] ZPush::GetSupportedProtocolVersions(): 12.0,12.1,14.0
05/09/2017 22:23:21 [27718] [DEBUG] [user] ZPush::GetSupportedCommands(): Sync,SendMail,SmartForward,SmartReply,GetAttachment,GetHierarchy,CreateCollection,DeleteCollection,MoveCollection,FolderSync,FolderCreate,FolderDelete,FolderUpdate,MoveItems,GetItemEstimate,MeetingResponse,ResolveRecipients,ValidateCert,Provision,Search,Ping,Notify,ItemOperations,Settings
05/09/2017 22:23:21 [27718] [ INFO] [user] Options request
05/09/2017 22:23:21 [27718] [ INFO] [user] cmd='' memory='1.70 MiB/2.00 MiB' time='0.54s' devType='' devId='' getUser='user' from='1.2.3.4' idle='0s' version='2.3.7+0' method='OPTIONS' httpcode='200'
05/09/2017 22:23:21 [27718] [DEBUG] [user] -------- End

At this point I am completely stumped. The iOS device and all the server-side software packages are up to date. There are no errors in the Apache logs or IMAP server (dovecot). I can’t find anyone else with the same problem. There don’t seem to be any tutorials online even on the basic setup of the IMAP backend. Before the upgrade to Stretch everything was working fine. Can anyone shed any light on this?

Thanks in advance!

[Sebastian]

How does your apache vhost config look like? It could be that there is some incompatible configuration.
Are you using mod_php?
Have a look at the z-push-config-apache that installs a vhost config.

[kitserve]

Hi Sebastian, thanks for your response. Below is the snippet from our Apache config. As you can see, it’s virtually identical to the one that comes with the z-push-config-apache package (that’s where we copied it from). We’re using mod_php7 from Debian Stretch.

# Z-Push / ActiveSync
Alias /Microsoft-Server-ActiveSync /usr/share/z-push/index.php
<Directory /usr/share/z-push>
	Require all granted
</Directory>

<Directory /usr/share/z-push>
	# Don't list a directory index, follow symlinks (maybe state dir is somewhere linked)
	DirectoryIndex index.php
	Options -Indexes +FollowSymLinks

	# Z-push requirements
	<IfModule mod_php7.c>
		php_value magic_quotes_gpc off
		php_value magic_quotes_runtime off
		php_value register_globals off
		php_value short_open_tag on

		# Optional
		# php_value display_errors off

		# Setting memory limit higher (larger attachments)
		php_value memory_limit 128M
	</IfModule>

	# Security
	# Don't allow .htaccess Overrides, disallow access to files
	AllowOverride none
	<IfModule !mod_authz_core.c>
		Order allow,deny
		allow from all
	</IfModule>
	<IfModule mod_authz_core.c>
		Require all granted
	</IfModule> 

	<Files "config.php">
		<IfModule !mod_authz_core.c>
			Deny from All
		</IfModule>
		<IfModule mod_authz_core.c>
			Require all denied
		</IfModule>
	</Files>
</Directory>

[kitserve]

I’ve made a small discovery on this: we used

define('IMAP_DEFAULTFROM', 'sql');

If I switch this back to the default of

define('IMAP_DEFAULTFROM', '');

then I’m able to set the account up. Here’s the SQL config:

define('IMAP_FROM_SQL_DSN', 'mysql:host=localhost;dbname=db;charset=utf8');
define('IMAP_FROM_SQL_USER', 'user');
define('IMAP_FROM_SQL_PASSWORD', 'pass');
define('IMAP_FROM_SQL_OPTIONS', serialize(array(PDO::ATTR_PERSISTENT => true)));
define('IMAP_FROM_SQL_QUERY', "SELECT identities.name, identities.email FROM identities, users WHERE users.username = '#username' AND identities.del = 0 AND identities.standard = 1 AND identities.user_id = users.user_id ORDER BY identities.identity_id LIMIT 1");
define('IMAP_FROM_SQL_FIELDS', serialize(array('name', 'email')));
define('IMAP_FROM_SQL_FROM', '#name <#email>');
define('IMAP_FROM_SQL_FULLNAME', '#name');

This configuration looks up information from a Roundcube identities table. I don’t know why it stopped working, as it worked before the upgrade to Stretch, and I’ve verified both the SQL login and query using phpMyAdmin. Unfortunately this makes BackendIMAP pretty much unusable on anything other than the most basic server that only handles email for one domain, as we need a way of automatically setting the from address for users sending mail.

[kitserve]

Okay, a bit more news. For reasons unknown, PHP no longer automatically sets the default port in the DSN. If I change the line to the one below then it works again:

define('IMAP_FROM_SQL_DSN', 'mysql:host=localhost;port=3306;dbname=db;charset=utf8');

Annoying, but it’s not Z-Push’s fault I suppose.

[kitserve]

Looks like I spoke too soon about this one. On one Debian Stretch server running PHP-FPM, I was able to get Z-Push working with the IMAP/combined backend. On the original server, running mod_php, it still doesn’t work. I’ve tried switching to just the IMAP backend and leaving IMAP_DEFAULTFROM blank, and it still fails with ‘Unable to verify account information’. A GET request to the ActiveSync URL in browser works. Nothing appears in /var/log/z-push.log, /var/log/z-push-error.log, /var/log/auth.log, /var/log/syslog or /var/log/mail.*.