Request for Feedback: New CalDAV and CardDAV implementation for Kopano
-
Hi All,
ich have the same problem here with german umlaut (ä, ö, ü) as mentioned by MichaelAnders2 and Mornsgrans with CardDAV.
If i check the kdav.log everything seems to be ok … the german umlauts are displayed correct with ä, ö, ü in the log file.
But when i have a look at the kdav created or updated contacts in webapp, deskapp, this letters are not displayed correctly. If i enter german ä, ö, ü, in webapp they are synced to the client with correct enoding.
i have kdav running on Debian 9 with kopano-core 8.7.5.
-
Hi @white-eagle,
which client are you using? Are the umlauts correct when you check the contact in that client?
Manfred
-
Hi Manfred,
sorry for the late answer.
I built my own CARDDAV client to sync contacts between kopano and our CRM (MS SQL database) automatically.
This kdav.log is for utf8 encoded vcard
The letter Ä was sent with hex codes 0xC3 0x8410.10.2019 12:54:34 [15759] [DEBUG] main ------------------ Start 10.10.2019 12:54:34 [15759] [DEBUG] main PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 10.10.2019 12:54:34 [15759] [DEBUG] main KDAV version master-ad05c96 10.10.2019 12:54:34 [15759] [DEBUG] main SabreDAV version 3.2.2 10.10.2019 12:54:34 [15759] [TRACE] dav KopanoSyncState->__construct(): Using db sqlite:/var/lib/kopano/kdav/syncstate.db 10.10.2019 12:54:34 [15759] [DEBUG] main KLogger->LogIncoming(): INPUT PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 HTTP/1.1 User-Agent: CardDav .NET/0.1 Content-Type: text/vcard Host: kdav.domain.local Content-Length: 143 Expect: 100-continue Authorization: Basic REDACTED BEGIN:VCARD VERSION:3.0 UID:3f016741-9ab8-46c3-9b9a-8694143389b8 N:A;Änderung;;; EMAIL;INTERNET:herbert@domain.at END:VCARD
At the end of this vcards logentry
HTTP/1.1 201 Created X-KDAV-Version: master-ad05c96 X-Sabre-Version: 3.2.2 X-Sabre-Ew-Gross: vCard validation warning: The FN property must appear in the VCARD component exactly 1 time Content-Length: 0
It think this is ok
But in DeskApp the name is displayed like this:
After this i tried sending a vcard with default encoding (knowing that this is not the right encoding) to be sure that i used utf8 before
10.10.2019 12:54:34 [15759] [DEBUG] main ------------------ Start 10.10.2019 12:54:34 [15759] [DEBUG] main PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 10.10.2019 12:54:34 [15759] [DEBUG] main KDAV version master-ad05c96 10.10.2019 12:54:34 [15759] [DEBUG] main SabreDAV version 3.2.2 10.10.2019 12:54:34 [15759] [TRACE] dav KopanoSyncState->__construct(): Using db sqlite:/var/lib/kopano/kdav/syncstate.db 10.10.2019 12:54:34 [15759] [DEBUG] main KLogger->LogIncoming(): INPUT PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 HTTP/1.1 User-Agent: CardDav .NET/0.1 Content-Type: text/vcard Host: kdav.domain.local Content-Length: 143 Expect: 100-continue Authorization: Basic REDACTED BEGIN:VCARD VERSION:3.0 UID:f57475d8-c48c-4b88-ba6a-0f219b581d0c N:A;▒nderung;;; EMAIL;INTERNET:herbert@domain.at END:VCARD
At the end of this vcard log
HTTP/1.1 201 Created X-KDAV-Version: master-ad05c96 X-Sabre-Version: 3.2.2 X-Sabre-Ew-Gross: vCard validation warning: Property is not valid UTF-8! A;▒nderung;;; Content-Length: 0
As you see … not valid UTF-8 is correct because i used default encoding.
In the first scenario i think i use correct utf8 encoded vCard but the letters are not correct when the contact is stored in kopano address book
-
I do also have a problem with encoding of Umlauts. I filed an issue here: https://github.com/Kopano-dev/kdav/issues/5#issuecomment-548848833
-
Hi @amaccuish,
there have been some fixes and also more logging. Please update to the latest master branch version.
@june @white-eagle thanks for reporting the umlaut issue. Unfortunately I didn’t have time to debug it more extensively yet.
Manfred
-
Hi Norman,
I am also experimenting with KDAV but somewhere I must have an error in its config.
But I always get the error message:
16.11.2019 19:49:14 [25224] [TRACE] dav KopanoDavBackend->Logon(): XXXXXXXXXXX / password
16.11.2019 19:49:14 [25224] [WARN ] error /var/www/kdav/lib/KopanoDavBackend.php:65 mapi_logon_zarafa(): Unable to setup service for provider (2)
16.11.2019 19:49:14 [25224] [INFO ] dav KopanoDavBackend->Logon(): Auth: ERROR - logon failed for user XXXXXXX
16.11.2019 19:49:14 [25224] [DEBUG] main KLogger->LogOutgoing(): OUTPUT:
HTTP/1.1 401 UnauthorizedCould someone here so kind and post me a working config of:
- the config for the virtual host
- the config.php
My KDAV runs in “/var/www/kdav” directory
I am configuring in circles ATM :-(
TIA
Uwe
-
Found the error…
Only 2 // in my statement of accessing the socket…
-
Card-DAV error(s) for a lot of addresses from Kopano :
******17.11.2019 18:12:03 [21232] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: 090e96b315d049b6b7b03a8aced71a4b7b7b06000000 - size: 349 - etag: “1574010712”
17.11.2019 18:12:03 [21232] [DEBUG] main KLogger->LogOutgoing(): OUTPUT:
HTTP/1.1 500 Internal Server Error
X-KDAV-Version: master-39e847e
X-Sabre-Version: 3.2.2
Content-Type: application/xml; charset=utf-8<?xml version=“1.0” encoding=“utf-8”?>
<d:error xmlns:d=“DAV:” xmlns:s=“http://sabredav.org/ns”>
<s:sabredav-version>3.2.2</s:sabredav-version>
<s:exception>Sabre\VObject\ParseException</s:exception>
<s:message>Invalid Mimedir file. Line starting at 13 did not follow iCalendar/vCard conventions</s:message>
</d:error>***17.11.2019 18:12:03 [21232] [DEBUG] main httpcode=‘500’ memory=‘2.49 MiB/4.00 MiB’ time=’***
Any hints what to do?
-
Hi @zash1958,
Search for “vcf generated by mapi_mapitovcf:” in the log for a contact with an issue and post that output. Without knowing what’s at line 13 it’s not possible to tell what’s wrong with the data.
Manfred
-
Here we have one, Name is changed
18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: 090e96b315d049b6b7b03a8aced71a4ba67806000000 - size: 326 - etag: "1574003873" 18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): addressBookId: principals/uherczeg:090e96b315d049b6b7b03a8aced71a4b570000000000 - cardUri: 090e96b315d049b6b7b$ 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetMapiFolder(): Id: principals/uherczeg:090e96b315d049b6b7b03a8aced71a4b570000000000 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetMapiMessageForId(): Searching for '090e96b315d049b6b7b03a8aced71a4ba57806000000' in 'principals/uherczeg:090e96b315d049b6b$ 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetMapiMessageForId(): Try PR_SOURCE_KEY 090e96b315d049b6b7b03a8aced71a4ba57806000000 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetIdOfMapiMessage(): Finding ID of Resource id #796 18.11.2019 17:54:11 [25294] [DEBUG] dav KopanoDavBackend->GetIdOfMapiMessage(): Found PR_SOURCE_KEY: 090e96b315d049b6b7b03a8aced71a4ba57806000000 18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): vcf generated by mapi_mapitovcf: BEGIN:VCARD VERSION:3.0 PRODID:-//Kopano//libicalmapi 8.7.5//EN N:Kasatschok;Andreas FN:Andreas Kasatschok ORG:KGB GmbH TEL;TYPE=MOBILE:+49 71359306180 UID:040000008200E00074C5B7101A82E0080000000080267F2E5A9DD5010000000000000000010000005844C138B19A40AF9EA20EA682BFEBD5 REV:2019-11-17T15:17:53Z END:VCARD 18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: 090e96b315d049b6b7b03a8aced71a4ba57806000000 - size: 320 - etag: "1574003873" 18.11.2019 17:54:11 [25294] [DEBUG] main KLogger->LogOutgoing(): OUTPUT: HTTP/1.1 500 Internal Server Error X-KDAV-Version: master-39e847e X-Sabre-Version: 3.2.2 Content-Type: application/xml; charset=utf-8 <?xml version="1.0" encoding="utf-8"?> <d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns"> <s:sabredav-version>3.2.2</s:sabredav-version> <s:exception>Sabre\VObject\ParseException</s:exception> <s:message>Invalid Mimedir file. Line starting at 13 did not follow iCalendar/vCard conventions</s:message> </d:error> 18.11.2019 17:54:11 [25294] [DEBUG] main httpcode='500' memory='2.27 MiB/4.00 MiB' time='2.18s' 18.11.2019 17:54:11 [25294] [DEBUG] main ------------------ End
-
Hi @zash1958,
it would be more helpful if you would provide the log as is without editing it. If I use the output from your log it works just fine. However the size of data is different in your log from what I see in mine, so I guess that comes from you editing the log.
If you have a valid Kopano subscription, you can open an issue with the support and send me the issue number so that I can look into it. Otherwise you could upload the log file or the relevant part of it and send me the link.
Manfred
-
Hello Manfred,
we have an active subscription. I will compress the log and send it to Kopano itself? Or send it it here to You?
TIA
Uwe Herczeg
-
Hi Uwe,
please send it to Kopano itself (attach the log to the KS issue).
Manfred
-
@Fux and @amaccuish,
the “Line starting at X did not follow iCalendar/vCard conventions” issue on zash1958’s system was caused by a contact with multi line additional information (NOTE property in vcard). Could you please check if you have such contacts and if possible to edit the additional information to be in one line and see if it solves the issue?
I’ve created https://jira.kopano.io/browse/KDAV-28 for this problem.
Manfred
-
Well sorry for the necroposting but I have created a couple of threads with no answers and saw this thread.
https://forum.kopano.io/topic/3913/kopano-kdav-not-syncing
https://github.com/Kopano-dev/kdav/issues/8I’m using kdav and have some issues. It seems that Mac Calendar (for macOS Big Sur) is not syncing well with kdav. After some tests it seems that the synchronisation between kdav and kopano-server is not working well. If you create an event in kopano-ical is not synced with mac calendar correctly. Sometimes attendees that you already invited just disappear from the event.
With thunderbird is working fine.
I see this errors in the logs:
30.11.2021 13:14:12 [ 5483] [ERROR] dav KopanoDavBackend->Sync(): Unable to get value from token: 0000000000 - folderId: principals/user1:f7758d40f8f24f568ae2382ce4232d1f480200000000 - /usr/share/kopano-kdav/lib/KopanoCardDavBackend.php:375 30.11.2021 13:14:13 [ 5395] [ERROR] dav KopanoDavBackend->Sync(): Unable to get value from token: 0000000000 - folderId: principals/user1:f7758d40f8f24f568ae2382ce4232d1f520200000000 - /usr/share/kopano-kdav/lib/KopanoCardDavBackend.php:375 30.11.2021 13:16:12 [ 5295] [ERROR] dav KopanoDavBackend->Sync(): Unable to get value from token: 0000000000 - folderId: principals/user1:f7758d40f8f24f568ae2382ce4232d1f480200000000 - /usr/share/kopano-kdav/lib/KopanoCardDavBackend.php:375 30.11.2021 13:16:13 [ 5275] [ERROR] dav KopanoDavBackend->Sync(): Unable to get value from token: 0000000000 - folderId: principals/user2:f7758d40f8f24f568ae2382ce4232d1f520200000000 - /usr/share/kopano-kdav/lib/KopanoCardDavBackend.php:375
03.12.2021 17:52:45 [ 35] [ERROR] cal KopanoCalDavBackend->getCalendarObject(): Object NOT FOUND - /usr/share/kopano-kdav/vendor/sabre/dav/lib/CalDAV/Calendar.php:165 03.12.2021 17:52:45 [ 35] [ERROR] cal KopanoCalDavBackend->getCalendarObject(): Object NOT FOUND - /usr/share/kopano-kdav/vendor/sabre/dav/lib/CalDAV/Calendar.php:109 03.12.2021 17:52:45 [ 35] [ERROR] cal KopanoCalDavBackend->getCalendarObject(): Object NOT FOUND - /usr/share/kopano-kdav/vendor/sabre/dav/lib/CalDAV/Calendar.php:165
Also I can see that MAC Calendar is creating random calendars like this:
10.97.150.37 - - [30/Nov/2021:09:56:16 +0000] "MKCALENDAR /kdav/calendars/user1/370C0949-B7D7-4E86-9402-9BA4ED79D159/ HTTP/1.1" 201 0 10.97.150.37 - - [30/Nov/2021:09:56:20 +0000] "MKCALENDAR /kdav/calendars/user1/1A15AE10-01B8-4748-A2CB-8E1BCD909DB6/ HTTP/1.1" 201 0 10.97.150.37 - - [30/Nov/2021:10:05:55 +0000] "MKCALENDAR /kdav/calendars/user1/D8D2BCC8-B903-4C3D-BFED-E4F77B86AAE2/ HTTP/1.1" 201 0 10.97.150.37 - - [30/Nov/2021:10:08:05 +0000] "MKCALENDAR /kdav/calendars/user1/981959DA-9ED8-4A29-A6A7-BDE6562EF7BC/ HTTP/1.1" 201 0
Do you guys have issues with mac calendar? I’m using latest version I think.
I’ve also tested ical and z-push and both works well. But ical in webapp has an issue with “all day” option as it takes 2 days instead only one.
-
Hi @irreleph4nt
years later i’m come back and try again 🙈
the installation of composer are done with version 2.3.6 on the www-data user.
root@noise:/var/www/kdav# su www-data -s /bin/bash www-data@noise:~/kdav$ cd /var/www/kdav www-data@noise:~/kdav$ ./composer update bash: ./composer: No such file or directory www-data@noise:~/kdav$ ./composer install bash: ./composer: No such file or directory www-data@noise:~/kdav$ ls composer.json composer.phar config.php lib log4php.xml phpunit.xml server.php TRADEMARKS version.php composer.lock config doc LICENSE mapi README.md tests vendor www-data@noise:~/kdav$ composer install Cannot create cache directory /var/www/.cache/composer/repo/https---repo.packagist.org/, or directory is not writable. Proceeding without cache Cannot create cache directory /var/www/.cache/composer/files/, or directory is not writable. Proceeding without cache Loading composer repositories with package information Installing dependencies (including require-dev) from lock file Package operations: 26 installs, 0 updates, 0 removals - Installing symfony/polyfill-ctype (v1.18.1): Downloading (100%) - Installing webmozart/assert (1.9.1): Downloading (100%) - Installing phpdocumentor/reflection-common (2.2.0): Downloading (100%) - Installing phpdocumentor/type-resolver (1.3.0): Downloading (100%) - Installing phpdocumentor/reflection-docblock (5.2.1): Downloading (100%) - Installing phpunit/php-token-stream (2.0.2): Downloading (100%) - Installing symfony/yaml (v4.4.13): Downloading (100%) - Installing sebastian/version (2.0.1): Downloading (100%) - Installing sebastian/resource-operations (1.0.0): Downloading (100%) - Installing sebastian/recursion-context (2.0.0): Downloading (100%) - Installing sebastian/object-enumerator (2.0.1): Downloading (100%) - Installing sebastian/global-state (1.1.1): Downloading (100%) - Installing sebastian/exporter (2.0.0): Downloading (100%) - Installing sebastian/environment (2.0.0): Downloading (100%) - Installing sebastian/diff (1.4.3): Downloading (100%) - Installing sebastian/comparator (1.2.4): Downloading (100%) - Installing phpunit/php-text-template (1.2.1): Downloading (100%) - Installing doctrine/instantiator (1.3.1): Downloading (100%) - Installing phpunit/phpunit-mock-objects (3.4.4): Downloading (100%) - Installing phpunit/php-timer (1.0.9): Downloading (100%) - Installing phpunit/php-file-iterator (1.4.5): Downloading (100%) - Installing sebastian/code-unit-reverse-lookup (1.0.1): Downloading (100%) - Installing phpunit/php-code-coverage (4.0.8): Downloading (100%) - Installing phpspec/prophecy (v1.10.3): Downloading (100%) - Installing myclabs/deep-copy (1.10.1): Downloading (100%) - Installing phpunit/phpunit (5.7.27): Downloading (100%) symfony/yaml suggests installing symfony/console (For validating YAML files using the lint command) sebastian/global-state suggests installing ext-uopz (*) phpunit/php-code-coverage suggests installing ext-xdebug (^2.5.1) phpunit/phpunit suggests installing ext-xdebug (*) phpunit/phpunit suggests installing phpunit/php-invoker (~1.1) Package phpunit/php-token-stream is abandoned, you should avoid using it. No replacement was suggested. Package phpunit/phpunit-mock-objects is abandoned, you should avoid using it. No replacement was suggested. Generating autoload files www-data@noise:~/kdav$ exit exit
but now i think my directive ist completly wrong
domain/kdav = blank site instead of asking user/password
caldav is working properlythis is at the end of the z-push config:
location /kdav { alias /var/www/kdav/; #root /var/www/kdav/; index server.php; # rewrite ^/.well-known/caldav /cal.php redirect; rewrite ^/.well-known/carddav /card.php redirect; rewrite ^/server.php$uri break; rewrite .*/server.php$uri break; rewrite .*/.well-known/carddav$ / redirect; # rewrite .*/.well-known/caldav$ / redirect; charset utf-8; location ~ ^(.+\.php)(.*)$ { fastcgi_split_path_info ^(.+\.php)(/.+)$; fastcgi_pass unix:/var/run/php/php7.2-fpm.sock; fastcgi_param SCRIPT_FILENAME $request_filename; fastcgi_index server.php; include fastcgi_params; } } location /caldav { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_pass http://127.0.0.1:8080; } }
Also i try a simpler directive:
location /kdav { root /var/www/kdav/; rewrite ^/.well-known/carddav$ / redirect; # rewrite ^/.well-known/caldav$ / redirect; rewrite ^(.*)$ /server.php break; } location /caldav { proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_pass http://127.0.0.1:8080; } }
This downloads the following file:
<?php /*********************************************** * File : server.php * Project : KopanoDAV * Descr : This is the entry point * through which all requests * are processed. * * Created : 13.12.2016 * * Copyright 2016 - 2018 Kopano b.v. * * This file is part of kDAV. kDAV is free software; you can redistribute * it and/or modify it under the terms of the GNU Affero General Public * License as published by the Free Software Foundation; either version 3 * or (at your option) any later version. * * This software uses SabreDAV, an open source software distributed * under three-clause BSD-license. Please see <http://sabre.io/dav/> * for more information about SabreDAV. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU Affero General Public License for more details. * * You should have received a copy of the GNU Affero General Public License * along with this program. If not, see <http://www.gnu.org/licenses/>. * * Consult LICENSE file for details ************************************************/ namespace Kopano\DAV; // require composer auto-loader require __DIR__ . '/vendor/autoload.php'; // Configure & create main logger KLogger::configure(__DIR__ . '/log4php.xml'); $logger = new KLogger('main'); // don't log any Sabre asset requests (images etc) if (isset($_REQUEST['sabreAction']) && $_REQUEST['sabreAction'] == 'asset') { $logger->resetConfiguration(); } // log the start data $logger->debug('------------------ Start'); $logger->debug('%s %s', $_SERVER['REQUEST_METHOD'], $_SERVER['REQUEST_URI']); $logger->debug('KDAV version %s', KDAV_VERSION); $logger->debug('SabreDAV version %s',\Sabre\DAV\Version::VERSION); $kdavBackend = new KopanoDavBackend(new KLogger(('dav'))); $authBackend = new AuthBasicBackend($kdavBackend); $authBackend->setRealm(SABRE_AUTH_REALM); $principalBackend = new PrincipalsBackend($kdavBackend); $kCarddavBackend = new KopanoCardDavBackend($kdavBackend, new KLogger('card')); $kCaldavBackend = new KopanoCalDavBackend($kdavBackend, new KLogger('cal')); // Setting up the directory tree $nodes = array( new \Sabre\DAVACL\PrincipalCollection($principalBackend), new \Sabre\CardDAV\AddressBookRoot($principalBackend, $kCarddavBackend), new \Sabre\CalDAV\CalendarRoot($principalBackend, $kCaldavBackend), ); // initialize the server $server = new \Sabre\DAV\Server($nodes); $server->setBaseUri(DAV_ROOT_URI); $server->setLogger(new KPSR3Logger($logger)); $authPlugin = new \Sabre\DAV\Auth\Plugin($authBackend, SABRE_AUTH_REALM); $server->addPlugin($authPlugin); // add our version to the headers $server->httpResponse->addHeader('X-KDAV-Version', KDAV_VERSION); // log the incoming request (only if authenticated) $logger->LogIncoming($server->httpRequest); $aclPlugin = new DAVACL(); $aclPlugin->allowUnauthenticatedAccess = false; $server->addPlugin($aclPlugin); $schedulePlugin = new KopanoSchedulePlugin($kdavBackend, new KLogger('schedule')); $server->addPlugin($schedulePlugin); $imipPlugin = new KopanoIMipPlugin($kdavBackend, new KLogger('imip')); $server->addPlugin($imipPlugin); $server->addPlugin(new \Sabre\CalDAV\ICSExportPlugin()); $server->addPlugin(new \Sabre\CardDAV\Plugin()); // TODO: do we need $caldavPlugin for anything? $caldavPlugin = new \Sabre\CalDAV\Plugin(); $server->addPlugin($caldavPlugin); if (strlen(SYNC_DB) > 0) { $server->addPlugin(new \Sabre\DAV\Sync\Plugin()); } if (DEVELOPER_MODE) { $server->addPlugin(new \Sabre\DAV\Browser\Plugin(false)); } $server->exec(); // Log outgoing data $logger->LogOutgoing($server->httpResponse); $logger->debug("httpcode='%s' memory='%s/%s' time='%ss'", http_response_code(), $logger->FormatBytes(memory_get_peak_usage(false)), $logger->FormatBytes(memory_get_peak_usage(true)), number_format(microtime(true) - $_SERVER["REQUEST_TIME_FLOAT"], 2)); $logger->debug('------------------ End');
Has anyone running kdav with nginx?