Z-Push with HA Zimbra Architecture . Proxy issue



  • Hello,

    we plan to migrate our actual zimbra installation from 8.0.7 OSE to 8.8.7 OSE. Our production is running on single zimbra server with z-push 2.1.3-1892 !
    In our preproduction environnement we have several servers like :
    1 HA Reverse-Proxy Cluster (HTTP requests)
    1 HA HAProxy Cluster (HTTP request, IMAP, POP, …)
    2 Zimbra Proxy server
    1 Z-Push Server for handle mobile request
    1 ZImbra mailbox server (the target is to have X mailbox server)

    The authentication is mapped on a CAS serveur :) (so the authentication is redirected on an url)

    Our 2.4.2+0 Z-Push server is installed on CentOS 7 for work with Zimbra Backend 68.
    Z-push is running but it cannot open a session , we always get this error
    05/06/2018 15:32:50 [ 2375] [ERROR] [alogin] Zimbra->Logon(): END Logon - Proxy Error { connected = false }
    05/06/2018 15:32:50 [ 2375] [FATAL] [alogin] Exception: (AuthenticationRequiredException) - Access denied. Proxy unable to initiate a session on user mailbox server
    05/06/2018 15:32:50 [ 2375] [ERROR] [alogin] TopCollector could not initialise IPC provider ‘’: No IPC provider available

    Our config.php file :
    define(‘TIMEZONE’, ‘Europe/Paris’);
    define(‘BASE_PATH’, dirname($_SERVER[‘SCRIPT_FILENAME’]). ‘/’);
    define(‘SCRIPT_TIMEOUT’, 0);
    define(‘USE_CUSTOM_REMOTE_IP_HEADER’, false);
    define(“CERTIFICATE_OWNER_PARAMETER”, “SSL_CLIENT_S_DN_CN”);
    define(‘USE_FULLEMAIL_FOR_LOGIN’, true);
    define(‘STATE_MACHINE’, ‘FILE’);
    define(‘STATE_DIR’, ‘/var/lib/z-push/’);
    define(‘IPC_PROVIDER’, ‘’);
    define(‘LOGBACKEND’, ‘filelog’);
    define(‘LOGLEVEL’, LOGLEVEL_DEBUG);
    define(‘LOGAUTHFAIL’, false);
    define(‘LOGUSERLEVEL’, LOGLEVEL_DEVICEID);
    define(‘LOGFILEDIR’, ‘/logs/z-push/’);
    define(‘LOGFILE’, LOGFILEDIR . ‘z-push.log’);
    define(‘LOGERRORFILE’, LOGFILEDIR . ‘z-push-error.log’);
    define(‘LOG_SYSLOG_HOST’, false);
    define(‘LOG_SYSLOG_PORT’, 514);
    define(‘LOG_SYSLOG_PROGRAM’, ‘z-push’);
    define(‘LOG_SYSLOG_FACILITY’, LOG_LOCAL0);
    define(‘PROVISIONING’, false);
    define(‘LOOSE_PROVISIONING’, false);
    define(‘PROVISIONING_POLICYFILE’, ‘policies.ini’);
    define(‘SYNC_CONFLICT_DEFAULT’, SYNC_CONFLICT_OVERWRITE_PIM);
    define(‘SYNC_FILTERTIME_MAX’, SYNC_FILTERTYPE_ALL);
    define(‘PING_INTERVAL’, 30);
    define(‘FILEAS_ORDER’, SYNC_FILEAS_LASTFIRST);
    define(‘SYNC_MAX_ITEMS’, 512);
    define(‘UNSET_UNDEFINED_PROPERTIES’, false);
    define(‘SYNC_CONTACTS_MAXPICTURESIZE’, 5242880);
    define(‘ALLOW_WEBSERVICE_USERS_ACCESS’, false);
    define(‘USE_PARTIAL_FOLDERSYNC’, false);
    define(‘PING_LOWER_BOUND_LIFETIME’, false);
    define(‘PING_HIGHER_BOUND_LIFETIME’, false);
    define(‘SYNC_TIMEOUT_MEDIUM_DEVICETYPES’, “SAMSUNGGTI”);
    define(‘SYNC_TIMEOUT_LONG_DEVICETYPES’, “iPod, iPad, iPhone, WP, WindowsOutlook, WindowsMail”);
    define(‘RETRY_AFTER_DELAY’, 300);
    define(‘BACKEND_PROVIDER’, ‘BackendZimbra’);
    define(‘SEARCH_PROVIDER’, ‘’);
    define(‘SEARCH_WAIT’, 10);
    define(‘SEARCH_MAXRESULTS’, 10);
    define(‘KOE_CAPABILITY_GAB’, true);
    define(‘KOE_CAPABILITY_RECEIVEFLAGS’, true);
    define(‘KOE_CAPABILITY_SENDFLAGS’, true);
    define(‘KOE_CAPABILITY_OOF’, true);
    define(‘KOE_CAPABILITY_OOFTIMES’, true);
    define(‘KOE_CAPABILITY_NOTES’, true);
    define(‘KOE_CAPABILITY_SHAREDFOLDER’, true);
    define(‘KOE_CAPABILITY_SENDAS’, true);
    define(‘KOE_CAPABILITY_SECONDARYCONTACTS’, true);
    define(‘KOE_CAPABILITY_SIGNATURES’, true);
    define(‘KOE_CAPABILITY_RECEIPTS’, true);
    define(‘KOE_CAPABILITY_IMPERSONATE’, true);
    define(‘KOE_GAB_STORE’, ‘SYSTEM’);
    define(‘KOE_GAB_FOLDERID’, ‘’);
    define(‘KOE_GAB_NAME’, ‘Z-Push-KOE-GAB’);

    Our backend/zimbra/config.php file :
    define(‘ZIMBRA_URL’, ‘http://proxy.domain.tld’);
    define(‘ZIMBRA_USER_DIR’,‘zimbra’);
    define(‘ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR’,5);
    define(‘ZIMBRA_DEBUG’,true);
    define(‘ZIMBRA_USER_DIR’,‘zimbra’);
    define(‘ZIMBRA_DISABLE_URL_OVERRIDE’, true);
    define(‘ZIMBRA_SMART_FOLDERS’,true);
    define(‘ZIMBRA_SYNC_CONTACT_PICTURES’, true);
    define(‘ZIMBRA_VIRTUAL_CONTACTS’,false);
    define(‘ZIMBRA_VIRTUAL_APPOINTMENTS’,false);
    define(‘ZIMBRA_VIRTUAL_TASKS’,false);
    define(‘ZIMBRA_VIRTUAL_NOTES’,false);
    define(‘ZIMBRA_IGNORE_EMAILED_CONTACTS’,true);
    define(‘ZIMBRA_HTML’,true);
    define(‘ZIMBRA_ENFORCE_VALID_EMAIL’,true);
    define(‘MBSTRING_OVERLOAD’, (extension_loaded(‘mbstring’) ? ini_get(‘mbstring.func_overload’) : false));

    I also tried to set the zimbra IP without success.

    What’s wrong ?
    thx for you help


  • Kopano

    Hi oloncle,

    I moved the topic to a more appropriate sub-forum. I’m not very familiar with Zimbra, but it sounds rather as Zimbra and not Z-Push issue.

    Manfred



  • Yes i think too, but the application architecture is complexe and I tried my luck anyway ^^

    After trouble shooting a bit, i able to say there is a missconfiguration on z-push. I tried to used the lastest version of z-push on the production and the result is the same.

    I always the IPC provider issue .

    05/06/2018 16:30:35 [ 2374] [FATAL] [olivier] Exception: (AuthenticationRequiredException) - Access denied. Proxy unable to initiate a session on user mailbox server
    05/06/2018 16:30:35 [ 2374] [ERROR] [olivier TopCollector could not initialise IPC provider ‘’: No IPC provider available

    In a default installation, the IPC provider is “embeded” but I got this error … how can I manage it ? I tried to use memcached provider but there is an issue, on CentOS 7, with the memcached-pecl extension … The RPM is unable to validate dependancies and your code is unable to use the extension from the system.

    long way :)



  • Which PHP version are you suing, and which PHP packages do you have installed?



  • Also, have you followed the advice in the INSTALL guide for whitelisting the Z-Push server in the zimbra DosFilter?

    To be sure, set the z-push logging level to WBXML, and in the zimbra config.php, set ZIMBRA_DEBUG to true. This will output all the SOAP requests/responses to the log file. Look for any HTML errors in the zimbra respoonses.

    It is also possible that your CAS integration is resulting in a HTML redirect instruction that the zimbra backend does not know how to handle.


  • Kopano

    Hi oloncle,

    @oloncle said in Z-Push with HA Zimbra Architecture . Proxy issue:

    I always the IPC provider issue .

    05/06/2018 16:30:35 [ 2374] [FATAL] [olivier] Exception: (AuthenticationRequiredException) - Access denied. Proxy unable to initiate a session on user mailbox server
    05/06/2018 16:30:35 [ 2374] [ERROR] [olivier TopCollector could not initialise IPC provider ‘’: No IPC provider available

    In a default installation, the IPC provider is “embeded” but I got this error … how can I manage it ? I tried to use memcached provider but there is an issue, on CentOS 7, with the memcached-pecl extension … The RPM is unable to validate dependancies and your code is unable to use the extension from the system.

    If you set up the Z-Push repository, you could just install z-push-ipc-sharedmemory package which will get the necessary dependencies for the default installation.

    The additional packages are also listed here: https://wiki.z-hub.io/display/ZP/Installation+from+source.

    Manfred



  • if you show your proxy acl, i might be able to see whats wrong.
    If i look at the 2 lines manfred posted, about the IPC, the i would say, you that you whitelisted your mail server, but your user auth is before the whitelist entry.

    check your proxy acl order.



  • My issue was coming from the IPC share memory. I re installed it and the IPC error has disappear but the proxy error was also present.
    I push from production the z-push version 2.1.3 and it “seems” work but in reality any account false ou true is working via z-push but the synchronisation is not working with a mobile phone. The data synchro is not available.

    ynchronized by user: test.user

    DeviceId: sec1925ecbacacce
    Device type: SamsungDevice
    UserAgent: Android-SAMSUNG-SM-A320FL/101.700
    Device Model: SM-A320FL
    Device IMEI: 357xx408xxx4xx9
    Device friendly name: a3y17ltexc
    Device OS: Android
    Device OS Language: français
    Device Phone nr: +33xxxxxxxx
    Device Operator: Free
    ActiveSync version: 14.0
    First sync: 2018-06-05 19:01
    Last sync: never
    Total folders: 1
    Synchronized folders: 0
    Synchronized data: None available
    Status: OK
    WipeRequest on: not set
    WipeRequest by: not set
    Wiped on: not set
    Attention needed: No errors known

    Anybody has already implemented z-push with Zimbra 8.8.7 ?



  • @liverpoolfcfan
    php-process-5.4.16-45.el7.x86_64
    php-pecl-memcache-3.0.8-4.el7.x86_64
    php-common-5.4.16-45.el7.x86_64
    php-mbstring-5.4.16-45.el7.x86_64
    php-soap-5.4.16-45.el7.x86_64
    php-pear-1.9.4-21.el7.noarch
    php-cli-5.4.16-45.el7.x86_64
    php-5.4.16-45.el7.x86_64
    php-xml-5.4.16-45.el7.x86_64



  • have you tried to change :

    define(‘ZIMBRA_URL’, ‘http://proxy.domain.tld’);
    

    to

    define(‘ZIMBRA_URL’, ‘https://proxy.domain.tld’);
    

    Please note, i dont know zimbra, but i did see: https://sourceforge.net/p/zimbrabackend/support-requests/167/
    and maybe you can use this info : https://forums.zimbra.org/viewtopic.php?t=60300#p270549



  • Yes already tried theses workaround.
    06/06/2018 11:04:48 [13350] [ INFO] [test.user] cmd=‘Provision’ memory=‘7.22 MiB/7.50 MiB’ time=‘0.12s’ devType=‘SamsungDevice’ devId=‘sec1925ecbacacce’ getUser=‘test.user’ from=‘X.X.X.X’ idle=‘0s’ version=‘2.4.2+0’ method=‘POST’ httpcode=‘401’

    We can see an 401 authentication error … We tried to manually send the soap request and it works :(



  • i also found this one.
    https://forum.kopano.io/topic/682/z-push-behind-basic-authentication
    check if this also applies to your setup.

    and check if your upgrade replaced a setting or more.

    /var/www/html/z-push/backend/zimbra/config.php
    

    check if you server is in the config.php

    For me, im out of thoughts… i do think this is a zimbra problem and it looks like zimbra is not detecting some z-push settings.



  • not my case the authentication is validated by a SSO. I unable a debug in the PHP code and see an error on the soap response .

    06/06/2018 12:12:29 [13606] [DEBUG] [test.user] Zimbra->SoapRequest(): SOAP Message: <soap:Envelope xmlns:soap=“http://www.w3.org/2003/05/soap-envelope”>
    soap:Header<context xmlns=“urn:zimbra”>
    <session />
    <authToken></authToken>
    <notify seq=“0” />
    <format type=“js” />
    <userAgent name=“Android-SAMSUNG-SM-A320FL/101.700(…acacce) devip=172.17.43.1 ZPZB” version=“68” />
    </context></soap:Header>
    soap:Body<NoOpRequest xmlns=“urn:zimbraMail” /></soap:Body>
    </soap:Envelope>
    06/06/2018 12:12:29 [13606] [DEBUG] [test.user] Zimbra->SoapRequest(): SOAP response: Erreur : SOAP content truncated 0!=687
    06/06/2018 12:12:29 [13606] [DEBUG] [test.user] NoOpResponse:
    06/06/2018 12:12:29 [13606] [DEBUG] [test.user] NoOpResponse:
    06/06/2018 12:12:29 [13606] [ERROR] [test.user] Zimbra->Logon(): END Logon - Proxy Error { connected = false }
    06/06/2018 12:12:29 [13606] [ INFO] [test.user] AuthenticationRequiredException: Access denied. Proxy unable to initiate a session on user mailbox server - code: 0 - file: /ideosante/data/z-push/backend/zimbra/zimbra.php:1208

    I think is a problem with the source code because even if i configure my preproduction z-push to communicate with my Zimbra server (8.0.7) in production, it’s doesn’t work. I got the same error.



  • A thing you could try also, is to disable gzip on the webserver running the z-push vhost.
    That might fix the SOAP content truncated 0!=687 part
    I’ve found some other things you can read, maybe it helps.

    Look at : https://github.com/iomarmochtar/ozpy maybe thats an option

    You could try to setup with https://xdebug.org/ a php debug extention.

    and these maybe related
    https://bugzilla.zimbra.com/show_bug.cgi?id=99482
    https://bugzilla.zimbra.com/show_bug.cgi?id=75827

    So i suggest, report this bug at zimba.



  • Re,

    after wide troubleshooting it’s should be an issue with Transfer-Encoding: chunked . The response size doesn’t match the attended size … We have tried all way rproxy, proxy, zimbra no luck :(
    Maybe a curl issue … troubleshooting again …



  • Got it !: The issue is due to a encoding problem, forcing a soap request in UTF-8 works … why the web server is not working correctly will be the last question !

    Come back asap with the workaround !



  • @oloncle said in Z-Push with HA Zimbra Architecture . Proxy issue:

    Anybody has already implemented z-push with Zimbra 8.8.7 ?

    I have set up a test instance of zimbra 8.8.8 on Centos 7, with z-push 2.4.2.beta1+0 and zimbra backend 68 running on Centos 6/Apache 2.4.latest and everything looks to work correctly.



  • @oloncle said in Z-Push with HA Zimbra Architecture . Proxy issue:

    Got it !: The issue is due to a encoding problem, forcing a soap request in UTF-8 works … why the web server is not working correctly will be the last question !

    Check your php.ini setting for the directive

    default_charset = “UTF-8”

    and your HTTPD settings for the directive

    AddDefaultCharset UTF-8



  • By the way, if you use RH/Centos with the sclo repositories, the php.ini file is not necessarily in the normal place - in my case the active one is found at /opt/etc/rh/rh-php70/php.ini

    Always a good idea when troubleshooting to first create a phpinfo.php file

    <?php
    phpinfo();
    ?>

    and see where it tells you it is picking up the php.ini file from.

    Configuration File (php.ini) Path /etc/opt/rh/rh-php70
    Loaded Configuration File /etc/opt/rh/rh-php70/php.ini



  • @oloncle said in Z-Push with HA Zimbra Architecture . Proxy issue:

    I think is a problem with the source code because even if i configure my preproduction z-push to communicate with my Zimbra server (8.0.7) in production, it’s doesn’t work. I got the same error.

    Did you edit the source zimbra.php file after downloading it?

    By default, all editing is done using Notepad++ with Encoding set to UTF-8. This should cause it to be interpreted by the web server as UTF-8, and to send requests/expect responses in UTF-8.

    If you edited the file with an editor that saved it in any other Encoding format then that could be the source of your issue.