Kopano Meet behind apache reverse proxy



  • Hi community,

    i have some trouble running kopano meet behind a reverse proxy.

    the following setup is wanted:

    internet <-> apache reverse proxy(https) <-> kwebd (https) <-> kwmserver

    Internal everything is working fine, because my pc is able to communicate with the virtual machine directly (over dns).

    Because we have only one public ip adress, we have to route our traffic through a virtual machine which is routing the subdomain to the corresponding virtual machine. unfortunatly this is not working for kopano meet.

    meet answers with a 400 http code ( i think the upgrade is not working correctly or apache is not able to route the wss://sub.domain.de/…)

    kwebd-request.log:

     "GET /api/kwm/v2/rtm/websocket/JJT8-F6S6qHfoWQG63HChDfIVTn5IOjz HTTP/1.1" 400 36 "-" 
    

    The web traffic machine is running with centos 7 and apache 2.4.6
    kopano virtual machine is running with ubuntu 16 and apache 2.4.18 and kopano-kwebd

    this is the regarding apache proxy config (which works fine for kopano mattermost running under http and ws):

    <IfModule mod_ssl.c>
    <VirtualHost *:443>
        ServerName sub.domain.de
        ServerAlias sub.domain.de
    
         SSLProxyEngine On
         SSLProxyVerify none
         SSLProxyCheckPeerCN off
         SSLProxyCheckPeerName off
    
        RewriteEngine Off
        RewriteCond %{HTTP:Connection} Upgrade [NC]
        RewriteCond %{HTTP:Upgrade} websocket [NC]
        RewriteRule .* wss://sub.domain.de/$1 [P,L]
        RewriteCond %{HTTP:Upgrade} !=websocket [NC]
        RewriteRule /(.*) https://sub.domain.de/$1 [P,L]
    
    
         ProxyPreserveHost On
         ProxyPass / https://sub.domain.de/
         ProxyPassReverse / https://sub.domain.de/
    

    Following version are installed:

    ii  kopano-backup                                   8.7.5.0-0+43.1                                           amd64        Utility to back up and restore Kopano stores
    ii  kopano-client                                   8.7.5.0-0+43.1                                           amd64        Kopano MAPI provider library
    ii  kopano-common                                   8.7.5.0-0+43.1                                           amd64        Shared files for Kopano services
    ii  kopano-contacts                                 8.7.5.0-0+43.1                                           amd64        MAPI provider adding contact folders in the addressbook
    ii  kopano-dagent                                   8.7.5.0-0+43.1                                           amd64        E-Mail Delivery Agent for Kopano Core
    ii  kopano-documentseditor                          5.4.2-0+3.1                                              amd64        LibreOffice Online WebSocket Daemon
    ii  kopano-gateway                                  8.7.5.0-0+43.1                                           amd64        POP3 and IMAP Gateway for Kopano Core
    ii  kopano-grapi                                    8.7.5.0-0+43.1                                           amd64        REST entrypoints to the Kopano Groupware Core stack
    ii  kopano-ical                                     8.7.5.0-0+43.1                                           amd64        ICal and CalDAV Gateway for Kopano Core
    ii  kopano-kapid                                    0.12.1-0+332.2                                           amd64        Kopano API HTTP REST-Endpoints
    ii  kopano-konnectd                                 0.25.1-0+337.1                                           amd64        Kopano Konnect OpenID Connect Provider service daemon
    ii  kopano-kwebd                                    0.8.0-0+297.1                                            amd64        Kopano Web Server
    ii  kopano-kwmserverd                               0.17.2-0+329.1                                           amd64        Kopano Web Meetings Server
    ii  kopano-lang                                     8.7.5.0-0+43.1                                           all          Translations for Kopano Core components
    ii  kopano-meet                                     1.0.1-0+339.1                                            all          Metapackage to install Kopano Meet
    ii  kopano-meet-packages                            1.0.1-0+339.1                                            all          Metapackage to install the entire Kopano Meet stack
    ii  kopano-meet-webapp                              1.0.1-0+339.1                                            all          Kopano Meet Webapp
    ii  kopano-migration-imap                           8.7.5.0-0+43.1                                           amd64        Utility to migrate between IMAP mailboxes
    ii  kopano-monitor                                  8.7.5.0-0+43.1                                           amd64        Quota Monitor for Kopano Core
    ii  kopano-python-utils                             8.7.5.0-0+43.1                                           amd64        Additional Python-based command-line utils for Kopano Core
    ii  kopano-python3-extras                           0.1.2+0-0+53.1                                           amd64        Kopano Python 3 extra dependencies
    ii  kopano-search                                   8.7.5.0-0+43.1                                           amd64        Indexed search engine for Kopano Core
    ii  kopano-server                                   8.7.5.0-0+43.1                                           amd64        Server component for Kopano Core
    ii  kopano-server-packages                          8.7.5.0-0+43.1                                           all          Metapackage to install the entire Kopano Core stack
    ii  kopano-spamd                                    8.7.5.0-0+43.1                                           amd64        ICS-driven spam learning daemon for Kopano/SpamAssassin
    ii  kopano-spooler                                  8.7.5.0-0+43.1                                           amd64        E-mail Spooler for Kopano Core
    ii  kopano-utils                                    8.7.5.0-0+43.1                                           amd64        Admin command-line utils for Kopano Core
    ii  kopano-webapp                                   3.5.10.2410+106.1                                        all          New and improved WebApp for Kopano
    ii  kopano-webapp-plugin-contactfax                 3.5.10.2410+106.1                                        all          Kopano WebApp fax plugin
    ii  kopano-webapp-plugin-desktopnotifications       2.0.3.26+32.1                                            all          Kopano WebApp Desktop notifications plugin
    ii  kopano-webapp-plugin-filepreviewer              2.2.0.26+24.1                                            all          Kopano File previewer plugin
    ii  kopano-webapp-plugin-files                      2.1.5.305+101.2                                          all          Adds Files functionality to Kopano enabling access to WebDAV and other files backends.
    ii  kopano-webapp-plugin-filesbackend-owncloud      2.1.0.87+42.5                                            all          Adds Owncloud specific functionality to Kopano Files plugin.
    ii  kopano-webapp-plugin-filesbackend-smb           2.1.0.50+31.5                                            all          Adds Samba specific functionality to Kopano Files plugin.
    ii  kopano-webapp-plugin-folderwidgets              3.5.10.2410+106.1                                        all          Kopano WebApp folder widgets plugin
    ii  kopano-webapp-plugin-gmaps                      3.5.10.2410+106.1                                        all          Kopano WebApp google maps plugin
    ii  kopano-webapp-plugin-htmleditor-minimal-tinymce 1.0.0.9+2.1                                              all          Kopano WebApp TinyMCE editor with minimal functionality
    ii  kopano-webapp-plugin-intranet                   1.0.0.4+16.1                                             all          This plugin adds one or more buttons in the top menu bar which can be used to open a webpage inside Kopano WebApp.
    ii  kopano-webapp-plugin-mattermost                 1.0+26.3                                                 all          Integrates Mattermost into WebApp
    ii  kopano-webapp-plugin-mdm                        2.1.1.109+38.1                                           all          Kopano WebApp MDM plugin
    ii  kopano-webapp-plugin-meetings                   3.0.6.34                                                 all          Kopano WebApp Meetings Plugin
    ii  kopano-webapp-plugin-pimfolder                  3.5.10.2410+106.1                                        all          Kopano WebApp personal inbox plugin
    ii  kopano-webapp-plugin-quickitems                 3.5.10.2410+106.1                                        all          Kopano WebApp quick items plugin
    ii  kopano-webapp-plugin-smime                      2.2.2.240+23.1                                           all          Kopano WebApp S/MIME plugin
    ii  kopano-webapp-plugin-spell                      2.0.0.23+41.1                                            all          Kopano WebApp Spellchecker plugin
    ii  kopano-webapp-plugin-spell-de-at                2.0.0.4+38.1                                             all          Kopano WebApp Spellchecker German (Austrian) dictionary plugin
    ii  kopano-webapp-plugin-spell-de-ch                2.0.0.5+38.1                                             all          Kopano WebApp Spellchecker German (Swiss) dictionary plugin
    ii  kopano-webapp-plugin-spell-de-de                2.0.0.3+38.1                                             all          Kopano WebApp Spellchecker German dictionary plugin
    ii  kopano-webapp-plugin-spell-en                   2.0.0.1+38.1                                             all          Kopano WebApp Spellchecker English dictionary plugin
    ii  kopano-webapp-plugin-spell-en-gb                2.0.0.1+38.1                                             all          Kopano WebApp Spellchecker English (GB) dictionary plugin
    ii  kopano-webapp-plugin-spell-es                   2.0.0.1+38.1                                             all          Kopano WebApp Spellchecker Spanish dictionary plugin
    ii  kopano-webapp-plugin-spell-fr                   2.0.0.1+38.1                                             all          Kopano WebApp Spellchecker French dictionary plugin
    ii  kopano-webapp-plugin-spell-it                   1.0.0+35.1                                               all          Kopano WebApp Spellchecker Italian dictionary plugin
    ii  kopano-webapp-plugin-spell-nl                   2.0.0.1+39.1                                             all          Kopano WebApp Spellchecker Dutch dictionary plugin
    ii  kopano-webapp-plugin-spell-pl-pl                2.0.0.0+39.1                                             all          Kopano WebApp Spellchecker Polish dictionary plugin
    ii  kopano-webapp-plugin-titlecounter               3.5.10.2410+106.1                                        all          Kopano WebApp Titlecounter plugin
    ii  kopano-webapp-plugin-webappmanual               3.5.10.2410+106.1                                        all          Kopano WebApp Manual plugin
    ii  kopano-webapp-plugin-zdeveloper                 3.5.10.2410+106.1                                        all          Kopano WebApp developer plugin
    ii  libgsoap-kopano-2.8.84                          2.8.84-0+3.1                                             amd64        Runtime libraries for gSOAP
    ii  libgsoap-kopano-2.8.86                          2.8.86-0+1.1                                             amd64        Runtime libraries for gSOAP
    ii  libvmime-kopano2                                0.9.2.85+7.1                                             amd64        Library for working with MIME messages and IMAP/POP/SMTP
    ii  php-kopano-smime                                1.0.00+4.1                                               amd64        PHP Kopano SMIME Extension extends the php-openssl functions.
    ii  python3-kopano                                  8.7.5.0-0+43.1                                           all          High-level Python 3 bindings for Kopano
    ii  python3-kopano-rest                             8.7.5.0-0+43.1                                           all          Kopano REST API bindings for Kopano for Python 3
    ii  python3-kopano-search                           8.7.5.0-0+43.1                                           all          Kopano search module for Python 3
    ii  python3-kopano-utils                            8.7.5.0-0+43.1                                           all          Kopano utils modules for Python 3
    ii  z-push-backend-kopano                           2.5.1+0-0                                                all          Z-Push Kopano backend
    ii  z-push-kopano                                   2.5.1+0-0                                                all          Z-Push for Kopano
    ii  z-push-kopano-gab2contacts                      2.5.1+0-0                                                all          GAB sync into a contacts folder for Kopano
    ii  z-push-kopano-gabsync                           2.5.1+0-0                                                all          GAB sync for Kopano
    
    

    every help is appreciated. thanks in advance

    ansib


  • Kopano

    400 is a bad request response. You are right in assuming that the most likely cause for this error reply on the websocket endpoint would be that the request is not having the proper websocket headers. You Apache configuration should do the trick though. Meet generally works fine behind Apache.

    For further debugging you could enable debug logging in kwmserver. With debug level it also logs errors which are caused by invalid client generated request data like the above case. That might help to find the issue.



  • This post is deleted!


  • @longsleep
    yeah you are right. following error log is appearing at kwmserverd:

    level=debug msg="websocket handshake error" error="websocket: the client is not using the websocket protocol: 'upgrade' token not found in 'Connection' header" manager=rtm
    

    something must be wrong in the apache config, because in the internal network everything is working fine (kwebd and kwmserver,…)

    maybe i should not upgrade the connection i the apache and the kwebd should do it? but if i change the config to this:

    SSLProxyEngine On
         SSLProxyVerify none
         SSLProxyCheckPeerCN off
         SSLProxyCheckPeerName off
    #### Kopano Meet ####
    
         ProxyPreserveHost On
    
         ProxyPassMatch "/api/kwm/v2/rtm/websocket/(.*)" "wss://sub.domain.de/api/kwm/v2/rtm/websocket/$1" nocanon
        ProxyPassReverse /api/kwm/v2/rtm/websocket/(.*) "wss://sub.domain.de/api/kwm/v2/rtm/websocket/$1"
    
    
         ProxyPreserveHost On
         ProxyPass / https://sub.domain.de/
         ProxyPassReverse / https://sub.domain.de/
    

    After this the connection to the kwmserverd is closed before fully established.


  • Kopano

    @ansib said in Kopano Meet behind apache reverse proxy:

    RewriteEngine Off
    

    This just jumped into my view. If that is in your real config then that is a problem.


  • Kopano

    @ansib said in Kopano Meet behind apache reverse proxy:

    ProxyPassMatch "/api/kwm/v2/rtm/websocket/(.)" “wss://sub.domain.de/api/kwm/v2/rtm/websocket/$1” nocanon
    ProxyPassReverse /api/kwm/v2/rtm/websocket/(.
    ) “wss://sub.domain.de/api/kwm/v2/rtm/websocket/$1”

    This config might work too but uses the https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html module to handle upgrade headers. For a better/faster config use something like:

    <Location "/api/kwm/v2/rtm/websocket/">
        ProxyPass "wss://sub.domain.de/api/kwm/v2/rtm/websocket/"
    </Location>
    

    and ensure, your server has the mod_proxy_wstunnel module enabled.

    When only ProxyPass rules are used in your config, you can have RewriteEngine Off.



  • @longsleep said in Kopano Meet behind apache reverse proxy:

    This just jumped into my view. If that is in your real config then that is a problem.

    you are right. this was a mistake while playing around with the config.

    this is the actual apache configuration:

    
         SSLProxyEngine On
         SSLProxyVerify none
         SSLProxyCheckPeerCN off
         SSLProxyCheckPeerName off
    
        RewriteEngine On
        RewriteCond %{HTTP:Connection} Upgrade [NC]
        RewriteCond %{HTTP:Upgrade} websocket [NC]
        RewriteRule (.*) wss://sub.domain.de/$1 [P,L]
        RewriteCond %{HTTP:Upgrade} !=websocket [NC]
        RewriteRule /(.*) https://sub.domain.de/$1 [P,L]
    
    #### Kopano Meet ####
    
        ProxyPreserveHost On
    
    <Location "/api/kwm/v2/rtm/websocket/">
        ProxyPass "wss://sub.domain.de/api/kwm/v2/rtm/websocket/"
    </Location>
    
    
    
    #### Kopano webapp ####
    
    
         ProxyPass / https://sub.domain.de/
         ProxyPassReverse / https://sub.domain.de/
    
    

    But the same problem still appears. (Bad requests by kwmserver)

    Does the kwebd service have problems running behind an apache proxy?

    kwebd is running under port 443 with tls enabled (with valid certificate) so apache <> kwebd is using an https connection.


  • Kopano

    @ansib said in Kopano Meet behind apache reverse proxy:

    Does the kwebd service have problems running behind an apache proxy?
    kwebd is running under port 443 with tls enabled (with valid certificate) so apache <> kwebd is using an https connection.

    No, kwebd is of no concern here since you said it works when used directly. So the issue is the Apache configuration. Not exactly sure where the issue is though.

    A working Apache configuration is like this

    RewriteEngine On
    RewriteCond %{HTTP:Connection} Upgrade [NC]
    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteRule /api/kwm/v2/(.*) ws://localhost:2015/api/kwm/v2/$1 [P,L]
    ProxyPass /api/kwm/v2/ http://localhost:2015/api/kwm/v2/ retry=0
    

    Since yours looks very similar - maybe another rule matches before those rules in your config.


Log in to reply