Isue with Kopano-Core 8.4.0.669 an z-push 2.3.6?

[ck0ne]

Hello forum,

I have one or more problems with copano core. I have migrated from ZCP 7.2.4 to Kopano 8.4.0.669. Until now it ran very well. I manually migrated the config. If I run the server with the kopano user / group, an authentication is not possible. If I run the server with the root user / group, I can authenticate via Webapp or z-Push.

Can someone explain to me what is wrong?

Here the corresponding logs from the z-push.log

Thu Jun  1 22:24:50 2017: [warning] Authentication by plugin failed for user "": Trying to authenticate failed: wrong username or password
Thu Jun  1 22:24:50 2017: [warning] Failed to authenticate user "user" from "file:///var/run/kopano/server.sock" using program "apache2"

and server.log

01/06/2017 13:10:05 [ 7849] [DEBUG] [user] -------- Start
01/06/2017 13:10:05 [ 7849] [DEBUG] [user] cmd='' devType='' devId='' getUser='user' from='IP' version='2.3.5' method='OPTIONS'
01/06/2017 13:10:05 [ 7849] [DEBUG] [user] ZPush::GetBackend(): trying autoload backend 'BackendKopano'
01/06/2017 13:10:05 [ 7849] [DEBUG] [user] BackendKopano using PHP-MAPI version: 8.4.0-669 - PHP version: 5.5.9-1ubuntu4.21
01/06/2017 13:10:05 [ 7849] [DEBUG] [user] KopanoBackend->Logon(): Trying to authenticate user 'user'..
01/06/2017 13:10:05 [ 7849] [WARN] [user] /var/www/z-push/backend/kopano/kopano.php:145 mapi_logon_zarafa(): Unable to setup service for provider (2)
01/06/2017 13:10:05 [ 7849] [ERROR] [user] KopanoBackend->Logon(): login failed with error code: 0xFFFFFFFF80040111
01/06/2017 13:10:05 [ 7849] [WARN] [user] KopanoBackend->Logon(): logon failed for user 'user'
01/06/2017 13:10:05 [ 7849] [ INFO] [user] AuthenticationRequiredException: Access denied. Username or password incorrect - code: 0 - file: /var/www/z-push/lib/request/requestprocessor.php:64
01/06/2017 13:10:05 [ 7849] [ INFO] [user] User-agent: 'unknown'
01/06/2017 13:10:05 [ 7849] [FATAL] [user] Exception: (AuthenticationRequiredException) - Access denied. Username or password incorrect
01/06/2017 13:10:05 [ 7849] [DEBUG] [user] TopCollector(): Initialized mutexid Resource id #20 and memid Resource id #21.
01/06/2017 13:10:05 [ 7849] [DEBUG] [user] TopCollector initialised with IPC provider 'IpcSharedMemoryProvider' with type '20'
01/06/2017 13:10:05 [ 7849] [ INFO] [user] cmd='' memory='1.62 MiB/2.00 MiB' time='0.02s' devType='' devId='' getUser='user' from='IP' version='2.3.5' method='OPTIONS' httpcode='401'
01/06/2017 13:10:05 [ 7849] [DEBUG] [user] -------- End

Directory /var/run/kopano

insgesamt 32
-rw-r--r-- 1 root   root   5 Jun  1 15:27 dagent.pid
-rw-r--r-- 1 kopano kopano 5 Jun  1 15:27 gateway.pid
-rw-r--r-- 1 kopano kopano 5 Jun  1 15:27 ical.pid
-rw-r--r-- 1 kopano kopano 5 Jun  1 15:27 monitor.pid
-rw-r--r-- 1 kopano kopano 5 Jun  1 15:27 presence.pid
-rw-rw-rw- 2 kopano kopano 0 Jun  1 15:27 presence.pid.lock
srw-rw---- 1 root   root   0 Jun  1 22:36 prio.sock
-rw-r--r-- 1 kopano kopano 5 Jun  1 15:27 search.pid
-rw-rw-rw- 2 kopano kopano 0 Jun  1 15:27 search.pid.lock
srwx------ 1 kopano kopano 0 Jun  1 15:27 search.sock
-rw-r--r-- 1 root   root   6 Jun  1 22:36 server.pid
srw-rw-rw- 1 root   root   0 Jun  1 22:36 server.sock
-rw-r--r-- 1 kopano kopano 5 Jun  1 15:27 spooler.pid
-rw-rw-rw- 2 kopano kopano 0 Jun  1 15:27 ubuntu.54c05740-1476
-rw-rw-rw- 2 kopano kopano 0 Jun  1 15:27 ubuntu.fdc00740-1404

server.cfg

##############################################################
# SERVER SETTINGS

# IP Address to bind to (empty for ANY)
# Set to ::1 or 127.0.0.1 if connections should only come from localhost
# and through the webserver proxy
#server_bind            =

# Accept normal TCP connections (not recommended to disable)
server_tcp_enabled      = yes

# Port to bind to
server_tcp_port         = 236

# Accept Unix pipe connections (not recommended to disable)
server_pipe_enabled     = yes

# Unix socket location
server_pipe_name        = /var/run/kopano/server.sock

# Priority Unix socket location
server_pipe_priority    = /var/run/kopano/prio.sock

# Name for identifying the server in a multi-server environment
server_name = Kopano

# Override the hostname of this server, used by Kerberos SSO if enabled
server_hostname =

# Database engine (mysql)
database_engine         = mysql

# Allow connections from normal users through the Unix socket
allow_local_users       = yes

# local admin users who can connect to any store (use this for the kopano-dagent)
# field is SPACE separated
# eg: local_admin_users = root vmail
local_admin_users       = sysadmin kopano

# The user has full rights on a folder by default, uncomment the following line to disable this.
# owner_auto_full_access = false
owner_auto_full_access = true

# e-mail address of the Kopano System user
system_email_address    = postmaster@localhost

# drop privileges and run the process as this user
run_as_user             = root
#run_as_user            = kopano
# drop privileges and run the process as this group
run_as_group            = root
#run_as_group           = kopano
# create a pid file for stopping the service via the init.d scripts
pid_file                = /var/run/kopano/server.pid

# run server in this path (when not using the -F switch)
#running_path = /var/lib/kopano

# Use given allocator library. Values like libtcmalloc.so.4,
# libtcmalloc_minimal.so.4 and libjemalloc.so.2 would work.
#allocator_library = default

# create memory coredumps upon crash in the running_path directory
coredump_enabled = yes

# session timeout for clients. Values lower than 300 will be upped to 300
# automatically. If the server hears nothing from a client in session_timeout
# seconds, then the session is killed.
session_timeout         = 300

# for temporary files
# consider mounting a `tmpfs' underneath this path (wherever you
# point it to)
tmp_path = /tmp

##############################################################
# LOG SETTINGS

# Logging method (syslog, file), syslog facility is 'mail'
log_method              = file

# Logfile (for log_method = file, '-' for stderr)
log_file                = /var/log/kopano/server.log

# Loglevel (0(none), 1(crit), 2(err), 3(warn), 4(notice), 5(info), 6(debug))
log_level               = 3

# Log timestamp - prefix each log line with timestamp in 'file' logging mode
log_timestamp           = 1

# Buffer logging in what sized blocks. 0 for line-buffered (syslog-style).
#log_buffer_size = 0

##############################################################
# AUDIT LOG SETTINGS

# Audit logging is by default not enabled
audit_log_enabled       = yes

# Audit logging method (syslog, file), syslog facility is 'authpriv'
audit_log_method        = syslog

# Audit logfile (for log_method = file, '-' for stderr)
audit_log_file          = /var/log/kopano/audit.log

# Audit loglevel (0=no logging, 1=full logging)
audit_log_level         = 1

# Audit log timestamp - prefix each log line with timestamp in 'file' logging mode
audit_log_timestamp     = 1

##############################################################
# MYSQL SETTINGS (for database_engine = mysql)

# MySQL hostname to connect to for database access
mysql_host              = localhost

# MySQL port to connect with (usually 3306)
mysql_port              = 3306

# The user under which we connect with MySQL
mysql_user              = user

# The password for the user (leave empty for no password)
mysql_password          = pass

# Override the default MySQL socket to access mysql locally
# Works only if the mysql_host value is empty or 'localhost'
mysql_socket            =

# Database to connect to
mysql_database          = zarafa

# Where to place attachments. Value can be 'database', 'files' or 's3'
attachment_storage      = files

# Enable fsync as method to make sure attachments are stored on disk where
# supported and will not be buffered by OS and/or filesystem. Please note
# this setting will lower attachment write performance depending on your
# environment but enhances data safety with disaster recovery.
# Only affects 'files' attachment storage backend.
attachment_files_fsync  = yes

# When attachment_storage is 'files', use this path to store the files
# When attachment_storage is 's3', use this path to set a prefix to all
# attachment data of a certain cluster, for example 'attach'
attachment_path         = /var/lib/kopano/attachments

# Compression level for attachments when attachment_storage is 'files'.
# Set compression level for attachments disabled=0, max=9
attachment_compression  = 6

##############################################################
# S3 STORAGE SETTINGS (for attachment_storage = s3)

# The hostname of the entry point to the S3 cloud where the bucket is located
# If you are using miniio or an other S3 compatible implementation that
# is using another port, you can specify the port with hostname:port.
#attachment_s3_hostname = s3-eu-west-1.amazonaws.com

# The region where the bucket is located
#attachment_s3_region = eu-west-1

# The protocol that should be used to connect to S3, 'http' or 'https' (preferred)
#attachment_s3_protocol = https

# The URL style of the bucket, "virtualhost" or "path"
#attachment_s3_uristyle = virtualhost

# The access key id of your S3 account
#attachment_s3_accesskeyid =

# The secret access key of your S3 account
#attachment_s3_secretaccesskey =

# The bucket name in which the files will be stored
#attachment_s3_bucketname =

##############################################################
#  SSL SETTINGS

# enable SSL support in server
server_ssl_enabled      = no

# Listen for SSL connections on this port
server_ssl_port         = 237

# Required Server certificate, contains the certificate and the private key parts
server_ssl_key_file     = /etc/kopano/ssl/server.pem

# Password of Server certificate
server_ssl_key_pass     = replace-with-server-cert-password

# Required Certificate Authority of server
server_ssl_ca_file      = /etc/kopano/ssl/cacert.pem

# Path with CA certificates, e.g. /etc/ssl/certs
server_ssl_ca_path      =

# SSL protocols to use, space-separated list of protocols
# (SSLv3 TLSv1 TLSv1.1 TLSv1.2); prefix with ! to lock out a protocol.
#server_ssl_protocols =

# SSL ciphers to use, set to 'ALL' for backward compatibility
server_ssl_ciphers = ALL:!LOW:!SSLv2:!EXP:!aNULL

# Prefer the server's order of SSL ciphers over client's
server_ssl_prefer_server_ciphers = no

# Path of SSL Public keys of clients
sslkeys_path            = /etc/kopano/sslkeys

##############################################################
# THREAD SETTINGS

# Number of server threads
# default: 8
threads                         =       8

# Watchdog frequency. The number of watchdog checks per second.
# default: 1
watchdog_frequency      =       1

# Watchdog max age. The maximum age in ms of a task before a
# new thread is started.
# default: 500
watchdog_max_age        =       500

# Maximum SOAP keep_alive value
# default: 100
server_max_keep_alive_requests  =       100

# SOAP recv timeout value (time between requests)
# default: 5
server_recv_timeout     =       5

# SOAP read timeout value (time during requests)
# default: 60
server_read_timeout     =       60

# SOAP send timeout value
# default: 60
server_send_timeout     =       60

##############################################################
#  OTHER SETTINGS

# Softdelete clean cycle (in days) 0=never running
softdelete_lifetime     = 30

# Sync lifetime, removes all changes remembered for a client after x days of inactivity
sync_lifetime           = 90

# Set to 'yes' if you have Kerberos or NTLM correctly configured for single sign-on
enable_sso = no

# Set to 'yes' if you want to show the GAB to your users
enable_gab = yes

# Authentication can be through plugin (default, recommended), pam or kerberos
auth_method = plugin

# If auth_method is set to pam, you should provide the pam service name
pam_service = passwd


#############################################################
# CACHE SETTINGS
#
# To see the live cache usage, use 'kopano-stats --system'.

# Size in bytes of the 'cell' cache (should be set as high as you can afford to set it)
cache_cell_size                         = 469M

# Size in bytes of the 'object' cache
#cache_object_size = 16M

# Size in bytes of the 'indexed object' cache
#cache_indexedobject_size = 32M

# Size in bytes of the userquota details
cache_quota_size                        = 1M

# Lifetime for userquota details
cache_quota_lifetime            = 1

# Size in bytes of the acl cache
cache_acl_size                          = 1M

# Size in bytes of the store id/guid cache
cache_store_size                        = 1M

# Size in bytes of the 'user id' cache (this is allocated twice)
cache_user_size                         = 1M

# Size in bytes of the 'user details' cache
cache_userdetails_size          = 25M

# Lifetime for user details
cache_userdetails_lifetime      = 0

# Size in bytes of the server details (multiserver setups only)
cache_server_size                       = 1M

# Lifetime for server details (multiserver setups only)
cache_server_lifetime   = 30


##############################################################
#  QUOTA SETTINGS

# The default Warning Quota Level. Set to 0 to disable this level.
# The user will receive an email when this level is reached. Value is in Mb. Default value is 0.
quota_warn              = 0

# The default Soft Quota Level. Set to 0 to disable this level.
# The user will still receive mail, but sending new mail is prohibited, until objects are removed from the store.
# VALUE is in Mb. Default value is 0.
quota_soft              = 0

# The default Hard Quota Level. Set to 0 to disable this level.
# The user can not receive and send mail, until objects are removed from the store.
# Value is in Mb. Default value is 0.
quota_hard              = 0

# The default Warning Quota Level for multitenant public stores. Set to 0 to disable this level.
# The tenant administrator will receive an email when this level is reached. Value is in Mb. Default value is 0.
companyquota_warn      = 0


##############################################################
#  USER PLUGIN SETTINGS

# Name of the plugin that handles users
# Required, default = db
# Values: ldap, unix, db, ldapms
user_plugin             = unix

# configuration file of the user plugin, examples can be found in /usr/share/doc/kopano/example-config
user_plugin_config      = /etc/kopano/unix.cfg

# location of the kopano plugins
# if you have a 64-bit distribution, this probably should be changed to /usr/lib64/kopano
plugin_path             = /usr/lib/kopano

# scripts which create stores for users from an external source
# used for ldap and unix plugins only
createuser_script               =       /etc/kopano/userscripts/createuser
deleteuser_script               =       /etc/kopano/userscripts/deleteuser
creategroup_script              =       /etc/kopano/userscripts/creategroup
deletegroup_script              =       /etc/kopano/userscripts/deletegroup
createcompany_script    =       /etc/kopano/userscripts/createcompany
deletecompany_script    =       /etc/kopano/userscripts/deletecompany

# Set this option to 'yes' to skip the creation and deletion of new users
# The action will be logged, so you can see if your changes to the plugin
# configuration are correct.
user_safe_mode = no

##############################################################
# MISC SETTINGS

# Thread size in KB, default is 512
# WARNING: Do not set too small, your server WILL crash
thread_stacksize = 512

# Enable multi-tenancy environment
# When set to true it is possible to create tenants within the
# kopano instance and assign all users and groups to particular
# tenants.
# When set to false, the normal single-tenancy environment is created.
enable_hosted_kopano = false

# Enable multi-server environment
# When set to true it is possible to place users and tenants on
# specific servers.
# When set to false, the normal single-server environment is created.
enable_distributed_kopano = false

# Display format of store name
# Allowed variables:
#  %u Username
#  %f Full name
#  %c Tenant's name
# default: %f
storename_format = %f

# Loginname format (for Multi-tenancy installations)
# When the user does not login through a system-wide unique
# username (like the email address) a unique name is created
# by combining the username and the tenantname.
# With this configuration option you can set how the
# loginname should be built up.
#
# Note: Do not use the = character in the format.
#
# Allowed variables:
#  %u Username
#  %c Teantname
#
# default: %u
loginname_format = %u

# Set to yes for Windows clients to be able to download the latest
# Kopano Outlook client from the storage server
client_update_enabled = false

# Place the correct Kopano Outlook Client in this directory for
# Windows clients to download through the storage server
client_update_path = /var/lib/kopano/client

# Recieve update information from the client (0 = disabled, 1 = only on error, 2 = log always)
client_update_log_level = 1

# Log location for the client auto update files
client_update_log_path = /var/log/kopano/autoupdate

# Everyone is a special internal group, which contains every user and group
# You may want to disable this group from the Global Addressbook by setting
# this option to 'yes'. Administrators will still be able to see the group.
hide_everyone = no

# System is a special internal user, which has super-admin privileges
# You may want to disable this user from the Global Addressbook by setting
# this option to 'yes'. Administrators will still be able to see the user.
hide_system = yes

# Use Indexing service for faster searching.
# Enabling this option requires the kopano-search service to
# be running.
search_enabled = yes

# Path to the kopano-search service, this option is only required
# if the server is going to make use of the indexing service.
search_socket = file:///var/run/kopano/search.sock

# Time (in seconds) to wait for a connection to the kopano-search service
# before terminating the indexed search request.
search_timeout = 10

# Allow enhanced ICS operations to speedup synchronization with cached profiles.
# default: yes
enable_enhanced_ics = yes

# SQL Procedures allow for some optimized queries when streaming with enhanced ICS.
# This is default disabled because you must set 'thread_stack = 256k' in your
# MySQL server config under the [mysqld] tag and restart your MySQL server.
enable_sql_procedures = no

# Synchronize GAB users on every open of the GAB (otherwise, only on
# kopano-admin --sync)
sync_gab_realtime = yes

# Disable features for users. This list is space separated.
# Currently valid values: imap pop3 mobile outlook
disabled_features = imap pop3

# Maximum number of deferred records in total
max_deferred_records = 0

# Maximum number of deferred records per folder
max_deferred_records_folder = 20

# Restrict the permissions that admins receive to folder permissions only. Please
# read the server.cfg manpage before enabling this option so you really understand
# the implications
restrict_admin_permissions = no

# The maximum level of attachment recursion; Defines the number of
# attachment-in-attachment in-attachment levels are allowed when saving and
# replicating objects in the database. If you really want a higher level of
# recursion than about 20, you probably have to increase MySQL's stack_size
# to allow replication to work properly.
embedded_attachment_limit = 20

# Header to detect whether a connection has been received through a proxy. The
# value of the header is not inspected. If the header exists then the connection
# is taken to be received via a proxy. An empty value disables proxy detection
# and the value of '*' is used to indicate that all connections are proxied
proxy_header =

# Allow searchfolder creation in shared stores
external_searchfolders = yes

The next problem is z-push. I do not know whether the two behave together or not. Also here I hope on a tip.

Since the migration, Z-Push has suppressed the sender information of some mails. I do not know this behavior at all. In the webapp the sender can be seen. So far the operation of the Zarafa and z-push was without problems.

Again, the corresponding z-push.log

01/06/2017 15:27:59 [ 1388] [WARN] [user] SyncObject->Check(): object from type SyncMail: parameter 'from' contains an invalid email address '"" <>'. Address is removed.
01/06/2017 15:27:59 [ 1388] [ INFO] [user] cmd='Sync' memory='2.79 MiB/3.25 MiB' time='0.15s' devType='iPhone' devId='devId' getUser='user' from='IP' version='2.3.5' method='POST' httpcode='200'

Thanks for helping

Carsten

[Sebastian]

How did you migrate? I am not sure about the login failure, but the issue with Z-Push 2.3.6. Did you upgrade to this version? Your log states 2.3.5.
You should also use the repositories. From your log I would say you were using the tarball.

Cheers,
Sebastian

[Coffee_is_life]

Hello @ck0ne,

check if kopano still knows the users:

kopano-admin -l

Second, check if z-push knows the users aswell

z-push-admin -a list

third, check the owner of pid-file andchange the start-user for all your kopano-services to root give us some information about your php version
last but not least, did you upgrade the webapp-mapi-version aswell?

ll /var/run/kopano/
insgesamt 36
-rw-r--r-- 1 root root 5 21. Mai 02:05 dagent.pid
-rw-r--r-- 1 root root 5 21. Mai 02:05 gateway.pid
-rw-r--r-- 1 root root 5 21. Mai 02:05 ical.pid
-rw-r--r-- 1 root root 5 21. Mai 02:05 monitor.pid
-rw-r--r-- 1 root root 5 21. Mai 02:30 presence.pid
-rw-rw-rw- 2 root root 0 21. Mai 02:30 presence.pid.lock
srw-rw---- 1 root root 0 21. Mai 02:05 prio.sock
-rw-r--r-- 1 root root 5 21. Mai 02:05 search.pid
-rw-rw-rw- 2 root root 0 21. Mai 02:05 search.pid.lock
srwx------ 1 root root 0 21. Mai 02:05 search.sock
-rw-r--r-- 1 root root 5 21. Mai 02:05 server.pid
srw-rw-rw- 1 root root 0 21. Mai 02:05 server.sock
-rw-r--r-- 1 root root 5 21. Mai 02:05 spooler.pid

Coffee_is_life

[ck0ne]

@Sebastian You ar right. Before I upgraded to Version 2.3.6 I ran version 2.3.5. It makes no difference. So I add the logs from this version. by now I have the repo Version 2.3.6.

[ck0ne]

@Coffee_is_life

if I execute the kopano-admin I got the following output:

# kopano-admin -l -vvv
[error  ] M4LMsgServiceAdmin::ConfigureMsgService() MSGServiceEntry failed 80040111: logon failed
[crit   ] CreateProfileTemp(): ConfigureMsgService failed 80040111: logon failed
[warning] CreateProfileTemp failed: 80040111: logon failed
Unable to open Admin session: logon failed (0x80040111)
Access was denied on default:.
Using the -v option (possibly multiple times) may give more hints.```

z-push show me the connected devices
```# z-push-admin -a list

All synchronized devices

Device id                           Synchronized users
-----------------------------------------------------
android1420070404301                user
6702f240b52e178477d0ca5825bb6e31    user
48eamq8nbl4mbc6odh6a3t6rs0          user

I use a fully patched Ubuntu14.04.05 LTS with php 5.5.9

What kind of web app-mapi? I got the tar from download.kopano.io and installed the contents by

dpkg -I kopano-webapp*

is there a missing file? I can’t see it in my directory.

Carsten

[Sebastian]

Is your kopano server up and running? Are there any messages in the kopano log? Which user backend do you use?

I think you are having some general issue. The Z-Push things could be just a symptom.

[ck0ne]

@Sebastian
I do not know how to continue. ZCP I have since the version 6.4 in operation. The server.log looks good, as long as I run the server as root. In z-push.log, there are sometimes warnings. This is mostly due to mobile loops.

As backend I use Webapp and Mobile devices with IOS and android.

By now the Server is up as root

[Coffee_is_life]

Hi @ck0ne,

i think the issue is with read/write permissions… if you see my post with the permissions on the socket and pid files.

in every config you can define a user with which this programm starts (run_as_user)
if possible, start all services with root or at least with the same

Coffee_is_life

[ck0ne]

@Coffee_is_life yes is see it. i’ve execute kopano as root. in my test environment it is ok. for a production it dont. how ca i drop down the privileges and run the server without issues?

[Coffee_is_life]

@ck0ne, i bet if every module (server, dagent, gateway, […]) is started with the same user

chown the log-folder (chown -R kopano:kopano /var/log/kopano/)

now the internal connections should work, but i cant test right now how this effects the apache2 user.

Coffee_is_life

[ck0ne]

@Coffee_is_life this is what i have initially done. Every module have been run as kopano. except the server because i’ve got the errors above.

[Coffee_is_life]

@ck0ne, pls post your …/z-push/backend/kopano/config.php

there should be something like

define('MAPI_SERVER', 'default:');

you use local socket or tcp connection?

Coffee_is_life