Documentation on kopano-web (kweb)

[longsleep]

@irreleph4nt said in Documentation on kopano-web (kweb):

If kweb can’t be used to also serve nextcloud, I’d be interested in understanding how I can make kweb serve everything related to kopano on a subdomain and have nginx do everything else web-related.

Well while kweb is primarily focused on providing an out of the box setup for common Kopano services, it has the option to add additional extra configuration via its --extra parameter. If you use the binscript, it is using an extra.conf by default (see https://stash.kopano.io/projects/KGOL/repos/kweb/browse/scripts/kopano-kwebd.binscript#19) if it can be found at that location.

That extra.conf can be used to add any additional configuration directives at the end of the built-in kopano site. It should be possible to add nextcloud and other stuff there.

Take a look at https://github.com/caddyserver/examples for inspiration.

Keep in mind, that when using kweb with the built-in kopano site, Nextcloud and others need to be in a sub path (like /nextcloud). A hack like this should do it:

extra.conf

# Additional directives for the kweb kopano site are added here.

} # This finishes the built-in Kopano site

# New site with a sub path.
example.com/nextcloud {
  # Add your Nextcloud directives here ...

# } # Do not finish the site, since kweb does that internally (automatically adds a } at the end)

As i said this is a hack (and it certainly looks like a hack too) and i do not guarantee that this will work for all times.

Any feedback is greatly appreciated.

[irreleph4nt]

Hey guys,

thank you for the swift and thorough replies! Based on the information you have given me, there are two setups that I am going to try over the weekend and probably next week:

1. use kweb with additional config for nextcloud (no nginx)

2. use nginx for everything non-kopano that I need and proxy kopano-related requests to kweb on a different port

For both options I already have a few questions!

@fbartels If I understand correctly what you have said above, when I use caddy mode I have to turn off tls for kweb entirely? Does this mode disable any other functionality / features of kweb that are worth mentioning and does kopano software (webapp, meet …) still work in this mode? I’d assume this mode however would solve the problem below. Does proxying to kweb have any negative impact on it serving everything I need for kopano to work?

@longsleep It looks like the approach you mentioned injects additional parameters into the kopano caddy vhost configuration. Is there a way to actually define another vhost alltogether and run it via kweb? I am thinking about this as I’d want PHP FPM / FastCGI as an example if not for kopano, then for other vhosts at least.

When I get to playing around with kweb, I’ll make sure to add some comments on my progress here.

Thanks again,
irreleph4nt

[longsleep]

@irreleph4nt said in Documentation on kopano-web (kweb):

@longsleep It looks like the approach you mentioned injects additional parameters into the kopano caddy vhost configuration. Is there a way to actually define another vhost alltogether and run it via kweb? I am thinking about this as I’d want PHP FPM / FastCGI as an example if not for kopano, then for other vhosts at least.

The example from earlier does just that. It ends the configuration for the kopano vhost, and starts a new one for nextcloud. Its a hackish way but works.

[irreleph4nt]

I missed the closing breaket just above the example.com string - my apologies. What it looked like to me was that everything would hence go into the same vhost but of course, you are right. This way a new vhost is added to the config kweb uses. Kinda neat, but hacky. :)

[fbartels]

@irreleph4nt said in Documentation on kopano-web (kweb):

when I use caddy mode I have to turn off tls for kweb entirely

that mas maybe easy to misunderstand. kweb has a native option to turn off tls. you do only need to use the “caddy mode” when you want to completely use your own configuration file. In which case you will no benefit from the builtin configuration for meet, konnect, …

@irreleph4nt said in Documentation on kopano-web (kweb):

Does proxying to kweb have any negative impact on it serving everything I need for kopano to work?

since you add another hop this will probably give a small performance penalty.

[irreleph4nt]

@fbartels Thank you for clearing up my misunderstanding. One last question, please:
Does kweb come with default configration for z-push built-in or do I need to add z-push configuartion myself, through the extra.conf file for example?

[fbartels]

@irreleph4nt no, there is no native configuration for z-push yet (only the legacy proxy option). If you have a working setup we are of course open to include it.

[irreleph4nt]

@fbartels @longsleep

So this is where my experiement with kweb ends. I am pretty disappointed because I can’t even get it to start (aka, “serve”). As you can see in below systemctl output, the “setup” command in ExecStart finishes successfully but the “serve” fails with a coredump.
In case you want to debug this, I’ll dump a truckload of info below, including my build script on Arch Linux, relevant filesystem hierarchy and systemctl output as well as a dump file.

OS: Arch Linux x64
Software: minimal base install as per ArchWiki: base group, openssh, network, time and locale config
Kopano packages (all compiled from source; git master): kopano-core, kopano-grapi, kopano-kapi, kopano-kdav, kopano-konnect, kopano-kweb, kopano-kwmserver, kopano-meet, kopano-webapp, kopano-webapp-plugins (all), libkcoidc, php-kopano-smime, z-push
kweb configuration: vanilla, no changes to .cfg file

Relevant folder hierarchy with all kopano packages installed:

[root@testbench coredump]# tree /etc/kopano
/etc/kopano
├── admin.cfg
├── archiver.cfg
├── autorespond.cfg
├── backup.cfg
├── dagent.cfg
├── gateway.cfg
├── grapi.cfg
├── ical.cfg
├── kapid.cfg
├── kdav.config.php
├── konnectd.cfg
├── konnectd-identifier-registration.yaml
├── konnectd-identifier-scopes.yaml
├── konnectkeys
├── kweb
│   └── extra.d
├── kwebd.cfg
├── kwmserverd.cfg
├── kwmserverd-registration.yaml
├── ldap.cfg
├── license
├── meet.config.json
├── migration-pst.cfg
├── monitor.cfg
├── msr.cfg
├── presence.cfg
├── quotamails
│   ├── companywarning.mail
│   ├── userhard.mail
│   ├── usersoft.mail
│   └── userwarning.mail
├── search.cfg
├── searchscripts
│   ├── attachments_parser
│   ├── attachments_parser.db
│   ├── xmltotext.xslt
│   └── zmktemp
├── server.cfg
├── spamd.cfg
├── spooler.cfg
├── statsd.cfg
├── unix.cfg
├── userscripts
│   ├── createcompany.d
│   ├── creategroup.d
│   ├── createuser.d
│   ├── deletecompany.d
│   ├── deletegroup.d
│   └── deleteuser.d
└── webapp
    ├── config.php
    ├── debug.php
    └── plugins
        ├── contactfax-config.php
        ├── desktopnotifications-config.php
        ├── filepreviewer-config.php
        ├── files-config.php
        ├── gmaps-config.php
        ├── intranet-config.php
        ├── mobile-device-management-config.php
        ├── pimfolder-config.php
        ├── smime-config.php
        ├── spellchecker-config.php
        ├── titlecounter-config.php
        ├── webappmanual-config.php
        └── zdeveloper-config.php

15 directories, 51 files


[root@testbench coredump]# LANG=C ls -alh /usr/share/webapps/
total 36K
drwxr-xr-x  7 root root 4.0K Mar 18 01:02 .
drwxr-xr-x 80 root root 4.0K Mar 18 02:07 ..
-rw-r--r--  1 http http 1.2K Mar 17 16:24 favicon.ico
drwxr-xr-x  6 http http 4.0K Mar 18 01:02 kopano-kdav
drwxr-xr-x  3 root root 4.0K Mar 18 01:02 kopano-konnect
drwxr-xr-x  3 http http 4.0K Mar 18 01:02 kopano-meet
drwxr-xr-x  5 root root 4.0K Mar 18 01:02 kopano-webapp
-rw-r--r--  1 http http   26 Mar 17 16:24 robots.txt
drwxr-xr-x  8 http http 4.0K Mar 18 01:02 z-push


[root@testbench coredump]# LANG=C ls -alh /var/log/kopano/
total 32K
drwxr-xr-x  8 root root 4.0K Mar 18 01:38 .
drwxr-xr-x 10 root root 4.0K Mar 18 01:02 ..
drwxr-xr-x  2 root root 4.0K Mar 17 01:36 kapi
drwxr-xr-x  2 http http 4.0K Mar 17 01:28 kdav
drwxr-xr-x  2 root root 4.0K Mar 17 01:30 konnect
drwxr-xr-x  2 root root 4.0K Mar 18 01:02 kopano-webapp-git
drwxr-xr-x  2 http http 4.0K Mar 17 16:24 kweb
-rw-r-----  1 http root    0 Mar 18 01:38 kwebd-requests.log
drwxr-xr-x  2 root root 4.0K Mar 15 12:58 kwmserver

Build script for kopano-kwebd:

# Maintainer: irreleph4nt
pkgname=kopano-kweb-git
_altname=kwebd
groups=('kopano-web' 'kopano')
pkgver=0.6.0.r0.g1a6298a
pkgrel=1
pkgdesc='Flexible web server with integrated URL routing for Kopano services.'
arch=('x86_64')
url='https://github.com/Kopano-dev/kweb'
license=('GPL')
conflicts=('kopano-kweb')
provides=('kopano-kweb')
makedepends=('go-pie' 'dep' 'git')
source=("kopano-kweb-git::git+https://github.com/Kopano-dev/kweb.git"
)

sha256sums=(	'SKIP'
)

pkgver() {
	cd "${pkgname}"

	( set -o pipefail
	git describe --long 2>/dev/null | sed 's/^v//;s/\([^-]*-g\)/r\1/;s/-/./g' ||
	printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
	)
}

prepare(){
	mkdir -p gopath/src/stash.kopano.io/kgol
	ln -rTsf ${pkgname} gopath/src/stash.kopano.io/kgol/${pkgname}
	export GOPATH="${srcdir}"/gopath
	
	cd ${srcdir}/${pkgname}
	# replace libexec in binscript with valid path
	sed -i -- 's|\/usr\/libexec\/|\/usr\/lib\/|' scripts/kopano-kwebd.binscript
	
	# replace WEB_ROOT in binscript
	sed -i -- 's|\/usr\/share\/kopano-kweb\/www|\/usr\/share\/webapps|' scripts/kopano-kwebd.binscript

	# replace STATICPWA_PATH in binscript
	sed -i -- 's|\/usr\/share\/kopano-|\/usr\/share\/webapps\/kopano-|' scripts/kopano-kwebd.binscript

	# replace http user / group in kwebd service file
	sed -i -- 's|User\=www-data|User\=http|' scripts/kopano-kwebd.service
	sed -i -- 's|Group\=www-data|Group\=http|' scripts/kopano-kwebd.service

	cd $GOPATH/src/stash.kopano.io/kgol/${pkgname}
	dep ensure
}

build() {
	export GOPATH="${srcdir}"/gopath
	export GOFLAGS="-gcflags=all=-trimpath=${PWD} -asmflags=all=-trimpath=${PWD} -ldflags=-extldflags=-zrelro -ldflags=-extldflags=-znow" 
	cd gopath/src/stash.kopano.io/kgol/${pkgname}

	make
}

package() {

	install -d -m 755 ${pkgdir}/usr/lib/sysusers.d/

	cat > ${pkgdir}/usr/lib/sysusers.d/kopano-kweb.conf <<--EndOfUserConfig
		#Type   Name    ID      GECOS   Home directory  Shell
		g       kopano  -       -
	-EndOfUserConfig

	cd gopath/src/stash.kopano.io/kgol/${pkgname}

	install -d -m 755 "${pkgdir}"/usr/lib/systemd/system/
	install -d -m 755 "${pkgdir}"/usr/lib/kopano/
	install -d -m 755 "${pkgdir}"/usr/bin/
	install -d -m 755 "${pkgdir}"/usr/share/doc/kopano/example-config/
	install -d -m 755 "${pkgdir}"/usr/share/webapps/
	install -d -m 755 "${pkgdir}"/etc/kopano/kweb/extra.d/
	install -d -m 755 "${pkgdir}"/etc/kopano/kweb/.kweb/
	install -d -m 755 "${pkgdir}"/var/log/kopano/kweb/
	cp bin/${_altname} "${pkgdir}"/usr/lib/kopano/${_altname}
	cp scripts/kopano-kwebd.service "${pkgdir}"/usr/lib/systemd/system/
	cp scripts/kwebd.cfg "${pkgdir}"/etc/kopano/
	cp scripts/kwebd.cfg "${pkgdir}"/usr/share/doc/kopano/example-config/
	cp scripts/kopano-kwebd.binscript "${pkgdir}"/usr/bin/kopano-kwebd
	cp scripts/robots.txt "${pkgdir}"/usr/share/webapps/
	cp scripts/favicon.ico "${pkgdir}"/usr/share/webapps/
	chown http:http ${pkgdir}/etc/kopano/kwebd.cfg
	chown -R http:http ${pkgdir}/var/log/kopano/
	chown -R http:http ${pkgdir}/usr/share/webapps/
	chmod +x "${pkgdir}"/usr/bin/kopano-kwebd
}

Top part of the kwebd.binscript after sed’s in PKGBUILD:

# Base defines.

EXE=/usr/lib/kopano/kwebd
DEFAULT_WEB_ROOT=/usr/share/webapps
DEFAULT_HTTPS_PORT=443
DEFAULT_HTTP_PORT=80
DEFAULT_HTTPS_PORT_ALTERNATE=9443
DEFAULT_HTTP_PORT_ALTERNATE=9080
DEFAULT_EXTRA_CONFIG=/etc/kopano/kweb/extra.d/extra.cfg
DEFAULT_ASSETS_PATH=/etc/kopano/kweb/.kweb
DEFAULT_STATICPWA_PATH_PATTERN=/usr/share/webapps/kopano-%name/%path

Output of systemctl status kopano-kwebd.service

[user@asrv kopano-repo]$ sudo systemctl status kopano-kwebd
● kopano-kwebd.service - Kopano Web Daemon
   Loaded: loaded (/usr/lib/systemd/system/kopano-kwebd.service; disabled; vendor preset: disabled)
   Active: failed (Result: core-dump) since Mon 2019-03-18 01:52:18 CET; 2s ago
  Process: 28528 ExecStartPre=/usr/sbin/kopano-kwebd setup (code=exited, status=0/SUCCESS)
  Process: 28529 ExecStart=/usr/sbin/kopano-kwebd serve (code=dumped, signal=TRAP)
 Main PID: 28529 (code=dumped, signal=TRAP)

Mär 18 01:52:18 asrv systemd[1]: kopano-kwebd.service: Service RestartSec=100ms expired, scheduling restart.
Mär 18 01:52:18 asrv systemd[1]: kopano-kwebd.service: Scheduled restart job, restart counter is at 5.
Mär 18 01:52:18 asrv systemd[1]: Stopped Kopano Web Daemon.
Mär 18 01:52:18 asrv systemd[1]: kopano-kwebd.service: Start request repeated too quickly.
Mär 18 01:52:18 asrv systemd[1]: kopano-kwebd.service: Failed with result 'core-dump'.
Mär 18 01:52:18 asrv systemd[1]: Failed to start Kopano Web Daemon.

Coredump:
I am not allowed to upload attachments to my post so I have placed the .lz4 on my selfhosted nextcloud here: link

Let me know if there is anything else I can provide.

Kind regards,
irreleph4nt

[longsleep]

Thanks for getting back to us. Unfortunately the core file is not really useful without the exact binary and even if i had it i doubt it would yield more information. You did not post the output of the actual command (like from systemd journal) - is that because it did not give any output?

From your build script i see that you use the systemd service file shipped with the source tree. I find it much more likely that the problem is t there since it uses systemd features like sandboxing and dropping of caps. Please check that the systemd service parameters there are compatible with your distro and the environment/kernel you are running inside.

[irreleph4nt]

@longsleep
Thanks for replying!

I have restarted the machine in question and issues another “systemctl start kopano-kwebd”, which produces the following in the journal, kwebd log files remain empty: link to bpaste (expires 2019-03-25 19:57:14).

Regarding the systemd service file, I am investigating this and asking for help on my distro’s channels. kwebd 's behaviour does not change though, even if I am calling the binscript directly with “/usr/sbin/kopano-kwebd serve”. The terminal only reads

Trace/Breakpoint triggered (core dump written)

EDIT: In hope this helps at all, I have uploaded another core dump as well as the binary to my nextcloud. See here: link to folder

[irreleph4nt]

EDIT:
The below is fixed. After some investigation and multiple re-compiles of kweb, the problem revealed was that your software does not support PIC/PIE (see Wikipedia for some additional background and why PIC/PIE is a good idea). Compared to my build script posted above, I have exchanged go-pie with go but left the GOFLAGS untouched. I have also tried the -buildmode=pie flag in combination with vanilla go, but that resulted in the same segfault as compiling with go-pie directly. My next post in this thread will be about kweb configuration.

@longsleep @fbartels

You probably don’t have to look into this much further. I have done something that on the surface seems very stupid: replacing my own binaries for kweb and konnect (which exhibited the same issues) with the pre-compiled ones from the community Debian packages. And … surprise … they “work”, i.e. both services start in a more or less operational state.

I must have done something wrong in my build script by following my distro’s go packaging guidelines. Unfortunately, I am not a go developer so I am having a hard time spotting what exactly causes my binaries to misbehave. Compared to your instructions on github, I have introduced only a very small number of changes:

1. I am calling dep ensure before I start the build
2. I have built towards go-pie instead of go to create secure binaries
3. I have introduced additional GOFLAGS to enable RELRO support and trimed the build path from binaries, all in one command:
   export GOFLAGS="-gcflags=all=-trimpath=${PWD} -asmflags=all=-trimpath=${PWD} -ldflags=-extldflags=-zrelro -ldflags=-extldflags=-znow"
   (this is obviously issued before the build itself starts)

If you can see anything in the above list that is likely to cause issues, please by all means shout. :)
I’ll keep tinkering with this for a bit, maybe I find something by accident?

[longsleep]

@irreleph4nt said in Documentation on kopano-web (kweb):

I must have done something wrong in my build script by following my distro’s go packaging guidelines. Unfortunately, I am not a go developer so I am having a hard time spotting what exactly causes my binaries to misbehave. Compared to your instructions on github, I have introduced only a very small number of changes:

Thanks for following up. I had a look and this is because kweb (and all most other Go software by Kopano) is built as self contained static binaries with zero external dependencies. If you look at the Makefile you will see that various additional flags are passed when building to create reproducable binaries with zero (not even libc) dependencies.

So your exact issue is caused by -extldflags -static which is used in the Makefile. This is not compatible with -buildmode pie and the resulting binary is broken since PIE relies on features of the external linker.

We might want to expose better control to the build parameters though environment variables in the future - i need to think about that a bit. For now, you have to patch the Makefile to get rid of the part which forces static self contained binaries if you want to build a PIE.

If you build with PIE you rely on the cgo runtime which is also mostly disabled for Kopano Go software for the same static binary reasons. So make sure to also set CGO_ENABLED=1 when building to avoid potential problems.

[fbartels]

@irreleph4nt said in Documentation on kopano-web (kweb):

replacing my own binaries for kweb and konnect (which exhibited the same issues) with the pre-compiled ones from the community Debian packages

you do not really need to extract these from the debian packages btw. you could also simply grab them from https://download.kopano.io/community/kweb:/ and https://download.kopano.io/community/konnect:/

[irreleph4nt]

@longsleep
Thank you so much for this detailed explanation! Once I have my setup running I’ll definitely return to the PIE problem and apply all the knowledge you have shared here. :)

@fbartels
You are right, I could have used the relevant tar files on the community server as well. That was probably my brain demanding a nap. Thanks for pointing it out!

I have hit another road block by now that might have to do with kweb, so I am shamelessly plugging it in this thread.

Issue:
https://FQDN/api/gc returns 404 not found even though kapid and grapi are running.

Steps taken:
Follow the KC Admin Manual all the way to page 28

• SSL certificate for the server [works]
• Configure and start kopano-server, including SSL, OID settings etc [works]
• configure and start kweb, including ACME [works]
• configure and test konnect [works]
• setup and start grapi [works]
• setup and start kapi [works]
• access /api/gc [fails]

When kapid is started, the journal is populated like this:

[root@testbench kapi-kvs]# systemctl status kopano-kapid
● kopano-kapid.service - Kopano API Daemon
   Loaded: loaded (/usr/lib/systemd/system/kopano-kapid.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-03-20 01:47:20 CET; 46min ago
  Process: 2051 ExecStartPre=/usr/sbin/kopano-kapid setup (code=exited, status=0/SUCCESS)
 Main PID: 2053 (kapid)
    Tasks: 10 (limit: 2371)
   Memory: 80.0M
   CGroup: /system.slice/kopano-kapid.service
           └─2053 /usr/lib/kopano/kapid serve --log-timestamp=false --listen=127.0.0.1:8039 --log-level=info --plugins-path=/usr/lib/kopano/kapi-plugins --iss=https://hostname.domain.de

Mär 20 02:32:11 testbench kopano-kapid[2053]: level=error msg="kvs: store initialize failed: failed to open database migration driver: attempt to write a readonly database"
Mär 20 02:32:21 testbench kopano-kapid[2053]: level=error msg="kvs: store initialize failed: failed to open database migration driver: attempt to write a readonly database"
Mär 20 02:32:31 testbench kopano-kapid[2053]: level=error msg="kvs: store initialize failed: failed to open database migration driver: attempt to write a readonly database"
Mär 20 02:32:41 testbench kopano-kapid[2053]: level=error msg="kvs: store initialize failed: failed to open database migration driver: attempt to write a readonly database"

All paths mentioned in the journal exist and are owned by kapi:kopano. At first start the error read something along the lines of “cannot open database kvs.db - file does not exist” so I have created an empty .db with sqlite kvn.db - but something seems to still be broken. The logs and journal however don’t provide any useful info to track the issue down.

For reference, below is my current kapid.cfg. All folders mentioned here exist and are owned by kapi:kopano. The DB migration scripts also reside where they need to.

##############################################################
# Kopano API SETTINGS

# OpenID Connect Issuer Identifier.
oidc_issuer_identifier= https://hostname.domain.de

# Address:port specifier for where kapid should listen for
# incoming connections.
listen = 127.0.0.1:8039

# Disable TLS validation for all client request.
# When set to yes, TLS certificate validation is turned off. This is insecure
# and should not be used in production setups.
#insecure = no

# Comman separated list of plugin names which should be loaded.
# If this is not set or the value is empty, kapid scans the plugins_path
# on startup and loads all plugins found.
#plugins =

# Path to the location of kapi plugins.
plugins_path = /usr/lib/kopano/kapi-plugins

###############################################################
# Log settings

# Log level controls the verbosity of the output log. It can be one of
# `panic`, `fatal`, `error`, `warn`, `info` or `debug`. Defaults to `info`.
log_level = info

###############################################################
# Groupware REST API (grapi) Plugin settings

# Path where to find Kopano Groupware REST (grapi) sockets.
plugin_grapi_socket_path = /var/run/kopano-grapi

###############################################################
# Pubs API (pubs) Plugin settings

# Path to a key file to be used as secret for Pubs HMAC tokens.
# If no secret_key file is set, a random value will be generated on
# startup (not suitable for production use, since it changes on
# restart). A suitable key file can be generated with
# `openssl rand -out /etc/kopano/kapid-pubs-secret.key -hex 64`.
#plugin_pubs_secret_key = /etc/kopano/kapid-pubs-secret.key

###############################################################
# Key value store API (kvs) Plugin settings

# Database backend to use for persistent storage of kvs data. A supported
# backend must be set (sqlite3, mysql). Defaults to `sqlite3` if not set.
plugin_kvs_db_drivername = sqlite3

# Database backend data source name. This setting depends on the storage
# backend (plugin_kvs_db_drivername). A DNS is required to use the kvs plugin.
# - For `sqlite3` the value should be the full path to the database file.
# - For `mysql`, us a MySQL DSN in the following format:
#   [username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]
#   See https://github.com/go-sql-driver/mysql#dsn-data-source-name for a
#   full list of supported MySQL DSN params with examples.
# If not set and plugin_kvs_db_drivername is also not set a default value will
# be used which uses SQLite3.
plugin_kvs_db_datasource = /var/lib/kopano/kapi-kvs/kvs.db

# Path where to find the database migration scripts.
plugin_kvs_db_migrations = /usr/lib/kopano/kapi-kvs/db/migrations

Second issue:
My distro does not store webapp files directly under /usr/share but in /usr/share/webapps instead. konnect works, so generally should not be a problem. What I can not figure out is how I can point kweb to the actual kopano webapp. Any hints, please? The following paths when appended to https://FQDN/ return 404 not found

• /webapp/
• /kopano-webapp/

/usr/share/webapps contains this:

[root@testbench kapi-kvs]# ls -alh /usr/share/webapps/
insgesamt 40K
drwxr-xr-x  7 http    http   4,0K 20. Mär 00:22 .
drwxr-xr-x 81 root    root   4,0K 20. Mär 00:22 ..
-rw-r--r--  1 http    http   1,2K 19. Mär 02:42 favicon.ico
-rw-r--r--  1 http    http    138 19. Mär 01:37 index.html
drwxr-xr-x  6 http    http   4,0K 20. Mär 00:22 kopano-kdav
drwxr-xr-x  3 konnect kopano 4,0K 20. Mär 00:22 kopano-konnect
drwxr-xr-x  3 http    http   4,0K 20. Mär 00:22 kopano-meet
drwxr-xr-x  5 http    http   4,0K 20. Mär 00:22 kopano-webapp
-rw-r--r--  1 http    http     26 19. Mär 02:42 robots.txt
drwxr-xr-x  8 http    http   4,0K 20. Mär 00:22 z-push

https://FQDN/ loads index.html which simply contains Hello World!.

[longsleep]

@irreleph4nt

@irreleph4nt said in Documentation on kopano-web (kweb):

Issue:
https://FQDN/api/gc returns 404 not found even though kapid and grapi are running.

This seems ok - why would it return something else? That URL is not a valid endpoint of the gc API. Try an endpoint with exists - for example /api/gc/v1/me (it will return 403 unless you pass a valid access token with bearer auth).

When kapid is started, the journal is populated like this:
urnal exist and are owned by kapi:kopano. At first start the error read something along the lines of “cannot open database kvs.db - file does not exist” so I have created an empty .db with sqlite kvn.db - but something seems to still be broken. The logs and journal however don’t provide any useful info to track the issue down.

For reference, below is my current kapid.cfg. All folders mentioned here exist and are owned by kapi:kopano. The DB migration scripts also reside where they need to.

plugin_kvs_db_datasource = /var/lib/kopano/kapi-kvs/kvs.db

Maybe this is related to systemd sandboxing. The kapi systemd service shipped with the source tree uses full isolation and drops all caps. From the systemd docs ProtectSystem=full should only make /usr /boot and /etc read-only though. So not exactly sure what is going on here.

Second issue:
My distro does not store webapp files directly under /usr/share but in /usr/share/webapps instead. konnect works, so generally should not be a problem. What I can not figure out is how I can point kweb to the actual kopano webapp. Any hints, please? The following paths when appended to https://FQDN/ return 404 not found

Good point, We should probably add this use case to the docs somewhere or even include it by default.

In the meanwhile, kweb has learned (0.6.0 and later) some additional configuration directives which make it easier to configure fastcgi apps like Kopano webapp. Use something like this:

alias /webapp/ /usr/share/webapp/
fastcgi2 /webapp/ 127.0.0.1:9000 php {
  without /webapp/
  root /usr/share/webapp/
}
folderish /webapp

Add these directives (and potentially others, as desired) to the extra.cfg which by default (if you use the binscript from the source tree) is loaded from /etc/kopano/kweb/extra.d/extra.cfg. Of course this requires a running php-fpm in this example at 127.0.0.1:9000. Unix sockets also would work. Take a look at the Caddy server fastcgi directive which is mostly the same as kweb’s fastcgi2.

[irreleph4nt]

@longsleep

Thank you very much for helping me figure all of this out. I really do appreciate it!
Meanwhile I have great news, as with your help I have gotten the following packages to work:

• kapi
• grapi
• konnect
• kweb
• kwmserver (and by extension, meet)

kweb makes setting all these up a very easy exercise and I am impressed by it’s performance so far! I have also collected some feedback which may be useful to you:

1. The “scope” files (i.e. the yaml files) that are used between some of the kopano packages, what they are needed for and how to set them up correctly is not documented anywhere. Readmes of several packages you provide mention that working examples should have been provided with the docs of the package but I have had a very hard time actually finding any files that did not just contain dummy data. Setting up some dedicated documentation for them would be very useful.
2. The endpoints and rewrites built into kweb are not documented which made it very hard to figure out how to access any of the webapps. Had you not helped me on the forums here, this would have been impossible to figure out. Documentation in general is very scarse when it comes to kweb.
3. Even though you do not support subdomains out of the box, making the ACME request you use for letsencrypt automation configurable would be a huge win.
4. The new grapi repository, or rather its Readme and requirements.txt, are not calling out all the python dependencies to make the software work. python-xapian for example is not mentioned.

The only thing remaining for now is kdav, which for the life of me I can not get to work with kweb or without. Using an alias directive similar to the ones you suggested before doesn’t work. Via the apache example config on github I was able to access the SabreDav landing page but even then there is a bug which prevents it’s usage:

The docs and config make you change the DAV URI but when set to the path that leads to kdav (i.e. /usr/share/webapps/kopano-kdav/), the example apache setup errors out after login prompt with something along the lines of The requested path '/' is outside of the base URI of '/usr/share/webapps/kopano-kdav/'which I assume happens due to the rewrite in the apache config. When the DAV URI is changed to /, I can get to the SabreDAV overview. However, trying to add a contact via CardDav in Evolution then comes back with an access denied error:

192.168.0.1 - testu [24/Mar/2019:23:12:22 +0100] "PUT /9321199165cf935314012bbf955dd0c94f13f18c.vcf HTTP/1.1" 403 330 "-" "Evolution/3.30.5"

[fbartels]

Hi @irreleph4nt,

great to hear that you got it running on Arch.

The documentation at the moment indeed focuses on getting stuff running based on our packages (which then auto create the needed files for you). Endpoints for Meet itself are documented in its manual. The endpoints of the rest api are indeed not explicitly documented (well they are actually mentioned in https://stash.kopano.io/projects/KC/repos/kapi/browse/README.md), but these are also not really relevant for the normal end user.

I don’t think grapi itself depends on python-xapian. this could come from python-kopano.

The best thing would be if you could open a pr with your proposed documentation addition (e.g. to the individual readme files). I am not quite sure if you pull your code directly out of our Bitbucket instance, but we also mirror our code to github (at https://github.com/kopano-dev) where you can easily open a pr (or an issue if something isn’t directly clear).

The only thing remaining for now is kdav

I would recommend to open a dedicated topic for this, Kdav has primarily been tested with the apple client suite, it may be an off standard behaviour of Evolution you are tripping over.

PS: not quite sure if I mentioned this before but a good learning piece are the docker images at https://github.com/zokradonh/kopano-docker. for konnect and kwmserver this is using the official docker containers, which means that a lot of the configuration is then done in the dockerfile or the accompanying startup script. kdav is also part of this setup.