Request for Feedback: New CalDAV and CardDAV implementation for Kopano
-
A few more tests:
Creating a task or calendar-entry leads to a duplication of each entry after sync.
If I delete one entry, the other remains in Kopano and needs to be removed manually. This leads to an error that it could not be found on server and I get the offer, to delete locally.
Attached the logfile in my dropbox (115KB) from creation to deletion.
https://www.dropbox.com/s/jmp3ja5bc5e2ktj/kdav.log.txt?dl=0Next issue:
The German “Umlauts” are transformed into a wrong charset after creating or modifying in emClient or Outlook 2007 with CalDAVSynchronizer.
“Rüdiger” becomes transformed to “Rüdiger”, if data are manipulated in Windows client (except Kopano Webapp).
After correcting the Umlauts in Kopano Webapp the become displayed corretly on mobile device, Kopano and emClient/Outlook.
It seems, that Umlauts become transformed on upload to UCS via Kdav, but they become trasferred in the right way from UCS to clients. -
@Fux said in Request for Feedback: New CalDAV and CardDAV implementation for Kopano:
When I try to access contacts via kdav I get the following error at a few contacts - the others work:
<?xml version="1.0" encoding="utf-8"?> <d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns"> <s:sabredav-version>3.2.2</s:sabredav-version> <s:exception>Sabre\VObject\ParseException</s:exception> <s:message>Invalid Mimedir file. Line starting at 19 did not follow iCalendar/vCard conventions</s:message> </d:error> 22.02.2019 14:50:13 [29577] [DEBUG] main httpcode='500' memory='1.25 MiB/2.00 MiB' time='0.06s' 22.02.2019 14:50:13 [29577] [DEBUG] main ------------------ End
Does anyone know where to search for a solution?
I have this issue. Has anyone submitted example contacts to demonstrate the problem? It currently means I can’t sync with Apple Contacts. The line where the problem is seems to be the UID line.
-
@amaccuish said in Request for Feedback: New CalDAV and CardDAV implementation for Kopano:
Has anyone submitted example contacts to demonstrate the problem?
I have not received any.
-
@fbartels I have just sent an email now!
-
Hi all
im stuck by a depreaching warning…using Ubuntu 18.04.3i do the following:
Install sabre-dav:
sudo apt-get install php-sabre-dav
create working kdav folder:
mkdir /var/www/kdav cd /var/www/kdav wget https://github.com/Kopano-dev/kdav/archive/v0.9.0.tar.gz tar xfvz v0.9.0.tar.gz mv kdav-0.9.0/* /var/www/kdav rm -r kdav-0.9.0 rm v0.9.0.tar.gz
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" php -r "if (hash_file('sha384', 'composer-setup.php') === 'a5c698ffe4b8e849a443b120cd5ba38043260d5c4023dbf93e1558871f1f07f58274fc6f4c93bcfd858c6bd0775cd8d1') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" php composer-setup.php php -r "unlink('composer-setup.php');"
setup the config php:
define('MAPI_SERVER', 'file:///var/run/kopano/server.sock'); // Authentication realm define('SABRE_AUTH_REALM', 'Kopano DAV'); // Location of the SabreDAV server. define('DAV_ROOT_URI', '/var/www/kdav/');
set rights:
chown -R root:www-data /var/www/kdav chmod 664 /var/www/kdav/config.php
Composer Install:
noise:/var/www/kdav$ sudo ./composer.phar install [sudo] password for noise: Deprecation warning: Your package name KopanoDAV is invalid, it should have a vendor name, a forward slash, and a package name. The vendor and package name can be words separated by -, . or _. The complete name should match "[a-z0-9]([_.-]?[a-z0-9]+)*/[a-z0-9]([_.-]?[a-z0-9]+)*". Make sure you fix this as Composer 2.0 will error. Do not run Composer as root/super user! See https://getcomposer.org/root for details Loading composer repositories with package information Updating dependencies (including require-dev) Package operations: 34 installs, 0 updates, 0 removals - Installing apache/log4php (2.3.0): Downloading (100%) - Installing psr/log (1.1.0): Downloading (100%) - Installing sabre/uri (1.2.1): Downloading (100%) - Installing sabre/event (3.0.0): Downloading (100%) - Installing sabre/http (v4.2.4): Downloading (100%) - Installing sabre/xml (1.5.1): Downloading (100%) - Installing sabre/vobject (4.2.0): Downloading (100%) - Installing sabre/dav (3.2.3): Downloading (100%) - Installing myclabs/deep-copy (1.9.3): Downloading (100%) - Installing sebastian/version (2.0.1): Downloading (100%) - Installing sebastian/resource-operations (1.0.0): Downloading (100%) - Installing sebastian/recursion-context (2.0.0): Downloading (100%) - Installing sebastian/object-enumerator (2.0.1): Downloading (100%) - Installing sebastian/global-state (1.1.1): Downloading (100%) - Installing sebastian/exporter (2.0.0): Downloading (100%) - Installing sebastian/environment (2.0.0): Downloading (100%) - Installing sebastian/diff (1.4.3): Downloading (100%) - Installing sebastian/comparator (1.2.4): Downloading (100%) - Installing symfony/polyfill-ctype (v1.12.0): Downloading (100%) - Installing symfony/yaml (v4.3.4): Downloading (100%) - Installing doctrine/instantiator (1.2.0): Downloading (100%) - Installing webmozart/assert (1.5.0): Downloading (100%) - Installing phpdocumentor/reflection-common (2.0.0): Downloading (100%) - Installing phpdocumentor/type-resolver (1.0.1): Downloading (100%) - Installing phpdocumentor/reflection-docblock (4.3.2): Downloading (100%) - Installing phpspec/prophecy (1.8.1): Downloading (100%) - Installing phpunit/php-text-template (1.2.1): Downloading (100%) - Installing phpunit/phpunit-mock-objects (3.4.4): Downloading (100%) - Installing phpunit/php-timer (1.0.9): Downloading (100%) - Installing sebastian/code-unit-reverse-lookup (1.0.1): Downloading (100%) - Installing phpunit/php-token-stream (2.0.2): Downloading (100%) - Installing phpunit/php-file-iterator (1.4.5): Downloading (100%) - Installing phpunit/php-code-coverage (4.0.8): Downloading (100%) - Installing phpunit/phpunit (5.7.27): Downloading (100%) sabre/vobject suggests installing hoa/bench (If you would like to run the benchmark scripts) sebastian/global-state suggests installing ext-uopz (*) symfony/yaml suggests installing symfony/console (For validating YAML files using the lint command) phpunit/php-code-coverage suggests installing ext-xdebug (^2.5.1) phpunit/phpunit suggests installing phpunit/php-invoker (~1.1) phpunit/phpunit suggests installing ext-xdebug (*) Package phpunit/phpunit-mock-objects is abandoned, you should avoid using it. No replacement was suggested. Writing lock file Generating autoload files
/var/www/kdav# php composer.phar Deprecation warning: Your package name KopanoDAV is invalid, it should have a vendor name, a forward slash, and a package name. The vendor and package name can be words separated by -, . or _. The complete name should match "[a-z0-9]([_.-]?[a-z0-9]+)*/[a-z0-9]([_.-]?[a-z0-9]+)*". Make sure you fix this as Composer 2.0 will error. Do not run Composer as root/super user! See https://getcomposer.org/root for details ______ / ____/___ ____ ___ ____ ____ ________ _____ / / / __ \/ __ `__ \/ __ \/ __ \/ ___/ _ \/ ___/ / /___/ /_/ / / / / / / /_/ / /_/ (__ ) __/ / \____/\____/_/ /_/ /_/ .___/\____/____/\___/_/ /_/ Composer version 1.9.0 2019-08-02 20:55:32
./phpunit tests PHPUnit 5.7.27 by Sebastian Bergmann and contributors. .EEEEE 6 / 6 (100%) Time: 112 ms, Memory: 6.00MB There were 5 errors: 1) Kopano\DAV\KopanoDavBackendTest::testConstruct PDOException: SQLSTATE[HY000] [14] unable to open database file /var/www/kdav/lib/KopanoSyncState.php:45 /var/www/kdav/lib/KopanoDavBackend.php:48 /var/www/kdav/tests/kdav/KopanoDavBackendTest.php:44 2) Kopano\DAV\KopanoDavBackendTest::testGetObjectIdFromObjectUri with data set #0 ('1234.ics', '.ics', '1234') PDOException: SQLSTATE[HY000] [14] unable to open database file /var/www/kdav/lib/KopanoSyncState.php:45 /var/www/kdav/lib/KopanoDavBackend.php:48 /var/www/kdav/tests/kdav/KopanoDavBackendTest.php:44 3) Kopano\DAV\KopanoDavBackendTest::testGetObjectIdFromObjectUri with data set #1 ('5678AF.vcf', '.vcf', '5678AF') PDOException: SQLSTATE[HY000] [14] unable to open database file /var/www/kdav/lib/KopanoSyncState.php:45 /var/www/kdav/lib/KopanoDavBackend.php:48 /var/www/kdav/tests/kdav/KopanoDavBackendTest.php:44 4) Kopano\DAV\KopanoDavBackendTest::testGetObjectIdFromObjectUri with data set #2 ('123400.vcf', '.ics', '123400.vcf') PDOException: SQLSTATE[HY000] [14] unable to open database file /var/www/kdav/lib/KopanoSyncState.php:45 /var/www/kdav/lib/KopanoDavBackend.php:48 /var/www/kdav/tests/kdav/KopanoDavBackendTest.php:44 5) Kopano\DAV\KopanoDavBackendTest::testGetObjectIdFromObjectUri with data set #3 ('1234.ics', '.vcf', '1234.ics') PDOException: SQLSTATE[HY000] [14] unable to open database file /var/www/kdav/lib/KopanoSyncState.php:45 /var/www/kdav/lib/KopanoDavBackend.php:48 /var/www/kdav/tests/kdav/KopanoDavBackendTest.php:44 ERRORS! Tests: 6, Assertions: 1, Errors: 5.
The other thing are the nginx directive (not shure if this are correct and can be working)
i want to implement it to the z-push directive like: https://smtp.domain.com/kdavi have a few uncommented… but can’t test because composer not running.
Edit: found a simpler install:
curl -sS https://getcomposer.org/installer | sudo php -- --install-dir=/usr/local/bin --filename=composer mkdir /var/www/kdav cd /var/www/kdav composer require sabre/dav ~3.2.0
nano /etc/nginx/sites-available/z-push.conf
location /kdav { root /var/www; index server.php; # rewrite ^(.*)$ /server.php; # rewrite ^ /kdav/server.php$uri; # rewrite ^/.well-known/caldav /cal.php redirect; # rewrite ^/.well-known/carddav /card.php redirect; # rewrite ^/server.php$uri break; # rewrite .*/server.php$uri break; # rewrite .*/.well-known/carddav$ / redirect; # rewrite .*/.well-known/caldav$ / redirect; # rewrite ^/.well-known/carddav$ / redirect; # rewrite ^/.well-known/caldav$ / redirect; # rewrite ^(.*)$ /server.php break; charset utf-8; location ~ ^/kdav/(.+\.php)(.*)$ { root /var/www/kdav; index server.php; fastcgi_split_path_info ^(.+\.php)(/.+)$; fastcgi_pass unix:/var/run/php/php7.2-fpm.sock; fastcgi_param SCRIPT_FILENAME $request_filename; fastcgi_index server.php; include fastcgi_params; } } }
wget https://github.com/Kopano-dev/kdav/archive/v0.9.0.tar.gz tar xfvz v0.9.0.tar.gz mv kdav-0.9.0/* /var/www/kdav rm -r kdav-0.9.0 rm v0.9.0.tar.gz
edit config.php
composer install
but run in the same:
Deprecation warning: Your package name KopanoDAV is invalid, it should have a vendor name, a forward slash, and a package name. The vendor and package name can be words separated by -, . or _. The complete name should match "[a-z0-9]([_.-]?[a-z0-9]+)*/[a-z0-9]([_.-]?[a-z0-9]+)*". Make sure you fix this as Composer 2.0 will error. Do not run Composer as root/super user! See https://getcomposer.org/root for details Loading composer repositories with package information Installing dependencies (including require-dev) from lock file Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies. Run update to update them. Nothing to install or update Generating autoload files
any hints are welcome ;-) thx
-
@noise
I think you might have a few things backwards here. First of all the depreciation warning itself is harmless, as it is a warning, not an error. So whilst it would be nice for a developer at Kopano to look at this, it won’t stop kdav from working.A few other observations:
- You are running composer install as root, which, as the console output tells you, you shouldn’t be doing. Instead, change to your webserver user and run composer with that.
su nginx -s /bin/bash cd /var/www/kdav ./composer update ./composer install
I am adding the composer update command because of the output you see in the terminal. I personally have never used that.
- The php dav extension you are installing in the beginning is unnecessary as composer downloads all dependencies to the kdav folder. You can hence skip that.
I myself am still struggling to get kdav working (again), so I’ll post back in case I succeed.
-
Hi All,
ich have the same problem here with german umlaut (ä, ö, ü) as mentioned by MichaelAnders2 and Mornsgrans with CardDAV.
If i check the kdav.log everything seems to be ok … the german umlauts are displayed correct with ä, ö, ü in the log file.
But when i have a look at the kdav created or updated contacts in webapp, deskapp, this letters are not displayed correctly. If i enter german ä, ö, ü, in webapp they are synced to the client with correct enoding.
i have kdav running on Debian 9 with kopano-core 8.7.5.
-
Hi @white-eagle,
which client are you using? Are the umlauts correct when you check the contact in that client?
Manfred
-
Hi Manfred,
sorry for the late answer.
I built my own CARDDAV client to sync contacts between kopano and our CRM (MS SQL database) automatically.
This kdav.log is for utf8 encoded vcard
The letter Ä was sent with hex codes 0xC3 0x8410.10.2019 12:54:34 [15759] [DEBUG] main ------------------ Start 10.10.2019 12:54:34 [15759] [DEBUG] main PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 10.10.2019 12:54:34 [15759] [DEBUG] main KDAV version master-ad05c96 10.10.2019 12:54:34 [15759] [DEBUG] main SabreDAV version 3.2.2 10.10.2019 12:54:34 [15759] [TRACE] dav KopanoSyncState->__construct(): Using db sqlite:/var/lib/kopano/kdav/syncstate.db 10.10.2019 12:54:34 [15759] [DEBUG] main KLogger->LogIncoming(): INPUT PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 HTTP/1.1 User-Agent: CardDav .NET/0.1 Content-Type: text/vcard Host: kdav.domain.local Content-Length: 143 Expect: 100-continue Authorization: Basic REDACTED BEGIN:VCARD VERSION:3.0 UID:3f016741-9ab8-46c3-9b9a-8694143389b8 N:A;Änderung;;; EMAIL;INTERNET:herbert@domain.at END:VCARD
At the end of this vcards logentry
HTTP/1.1 201 Created X-KDAV-Version: master-ad05c96 X-Sabre-Version: 3.2.2 X-Sabre-Ew-Gross: vCard validation warning: The FN property must appear in the VCARD component exactly 1 time Content-Length: 0
It think this is ok
But in DeskApp the name is displayed like this:
After this i tried sending a vcard with default encoding (knowing that this is not the right encoding) to be sure that i used utf8 before
10.10.2019 12:54:34 [15759] [DEBUG] main ------------------ Start 10.10.2019 12:54:34 [15759] [DEBUG] main PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 10.10.2019 12:54:34 [15759] [DEBUG] main KDAV version master-ad05c96 10.10.2019 12:54:34 [15759] [DEBUG] main SabreDAV version 3.2.2 10.10.2019 12:54:34 [15759] [TRACE] dav KopanoSyncState->__construct(): Using db sqlite:/var/lib/kopano/kdav/syncstate.db 10.10.2019 12:54:34 [15759] [DEBUG] main KLogger->LogIncoming(): INPUT PUT /addressbooks/abcuser/Kontakte/3729D691-2A66A8B8-EEA4DA27 HTTP/1.1 User-Agent: CardDav .NET/0.1 Content-Type: text/vcard Host: kdav.domain.local Content-Length: 143 Expect: 100-continue Authorization: Basic REDACTED BEGIN:VCARD VERSION:3.0 UID:f57475d8-c48c-4b88-ba6a-0f219b581d0c N:A;▒nderung;;; EMAIL;INTERNET:herbert@domain.at END:VCARD
At the end of this vcard log
HTTP/1.1 201 Created X-KDAV-Version: master-ad05c96 X-Sabre-Version: 3.2.2 X-Sabre-Ew-Gross: vCard validation warning: Property is not valid UTF-8! A;▒nderung;;; Content-Length: 0
As you see … not valid UTF-8 is correct because i used default encoding.
In the first scenario i think i use correct utf8 encoded vCard but the letters are not correct when the contact is stored in kopano address book
-
I do also have a problem with encoding of Umlauts. I filed an issue here: https://github.com/Kopano-dev/kdav/issues/5#issuecomment-548848833
-
Hi @amaccuish,
there have been some fixes and also more logging. Please update to the latest master branch version.
@june @white-eagle thanks for reporting the umlaut issue. Unfortunately I didn’t have time to debug it more extensively yet.
Manfred
-
Hi Norman,
I am also experimenting with KDAV but somewhere I must have an error in its config.
But I always get the error message:
16.11.2019 19:49:14 [25224] [TRACE] dav KopanoDavBackend->Logon(): XXXXXXXXXXX / password
16.11.2019 19:49:14 [25224] [WARN ] error /var/www/kdav/lib/KopanoDavBackend.php:65 mapi_logon_zarafa(): Unable to setup service for provider (2)
16.11.2019 19:49:14 [25224] [INFO ] dav KopanoDavBackend->Logon(): Auth: ERROR - logon failed for user XXXXXXX
16.11.2019 19:49:14 [25224] [DEBUG] main KLogger->LogOutgoing(): OUTPUT:
HTTP/1.1 401 UnauthorizedCould someone here so kind and post me a working config of:
- the config for the virtual host
- the config.php
My KDAV runs in “/var/www/kdav” directory
I am configuring in circles ATM :-(
TIA
Uwe
-
Found the error…
Only 2 // in my statement of accessing the socket…
-
Card-DAV error(s) for a lot of addresses from Kopano :
******17.11.2019 18:12:03 [21232] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: 090e96b315d049b6b7b03a8aced71a4b7b7b06000000 - size: 349 - etag: “1574010712”
17.11.2019 18:12:03 [21232] [DEBUG] main KLogger->LogOutgoing(): OUTPUT:
HTTP/1.1 500 Internal Server Error
X-KDAV-Version: master-39e847e
X-Sabre-Version: 3.2.2
Content-Type: application/xml; charset=utf-8<?xml version=“1.0” encoding=“utf-8”?>
<d:error xmlns:d=“DAV:” xmlns:s=“http://sabredav.org/ns”>
<s:sabredav-version>3.2.2</s:sabredav-version>
<s:exception>Sabre\VObject\ParseException</s:exception>
<s:message>Invalid Mimedir file. Line starting at 13 did not follow iCalendar/vCard conventions</s:message>
</d:error>***17.11.2019 18:12:03 [21232] [DEBUG] main httpcode=‘500’ memory=‘2.49 MiB/4.00 MiB’ time=’***
Any hints what to do?
-
Hi @zash1958,
Search for “vcf generated by mapi_mapitovcf:” in the log for a contact with an issue and post that output. Without knowing what’s at line 13 it’s not possible to tell what’s wrong with the data.
Manfred
-
Here we have one, Name is changed
18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: 090e96b315d049b6b7b03a8aced71a4ba67806000000 - size: 326 - etag: "1574003873" 18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): addressBookId: principals/uherczeg:090e96b315d049b6b7b03a8aced71a4b570000000000 - cardUri: 090e96b315d049b6b7b$ 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetMapiFolder(): Id: principals/uherczeg:090e96b315d049b6b7b03a8aced71a4b570000000000 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetMapiMessageForId(): Searching for '090e96b315d049b6b7b03a8aced71a4ba57806000000' in 'principals/uherczeg:090e96b315d049b6b$ 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetMapiMessageForId(): Try PR_SOURCE_KEY 090e96b315d049b6b7b03a8aced71a4ba57806000000 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetStore(): storename uherczeg 18.11.2019 17:54:11 [25294] [TRACE] dav KopanoDavBackend->GetIdOfMapiMessage(): Finding ID of Resource id #796 18.11.2019 17:54:11 [25294] [DEBUG] dav KopanoDavBackend->GetIdOfMapiMessage(): Found PR_SOURCE_KEY: 090e96b315d049b6b7b03a8aced71a4ba57806000000 18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): vcf generated by mapi_mapitovcf: BEGIN:VCARD VERSION:3.0 PRODID:-//Kopano//libicalmapi 8.7.5//EN N:Kasatschok;Andreas FN:Andreas Kasatschok ORG:KGB GmbH TEL;TYPE=MOBILE:+49 71359306180 UID:040000008200E00074C5B7101A82E0080000000080267F2E5A9DD5010000000000000000010000005844C138B19A40AF9EA20EA682BFEBD5 REV:2019-11-17T15:17:53Z END:VCARD 18.11.2019 17:54:11 [25294] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: 090e96b315d049b6b7b03a8aced71a4ba57806000000 - size: 320 - etag: "1574003873" 18.11.2019 17:54:11 [25294] [DEBUG] main KLogger->LogOutgoing(): OUTPUT: HTTP/1.1 500 Internal Server Error X-KDAV-Version: master-39e847e X-Sabre-Version: 3.2.2 Content-Type: application/xml; charset=utf-8 <?xml version="1.0" encoding="utf-8"?> <d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns"> <s:sabredav-version>3.2.2</s:sabredav-version> <s:exception>Sabre\VObject\ParseException</s:exception> <s:message>Invalid Mimedir file. Line starting at 13 did not follow iCalendar/vCard conventions</s:message> </d:error> 18.11.2019 17:54:11 [25294] [DEBUG] main httpcode='500' memory='2.27 MiB/4.00 MiB' time='2.18s' 18.11.2019 17:54:11 [25294] [DEBUG] main ------------------ End
-
Hi @zash1958,
it would be more helpful if you would provide the log as is without editing it. If I use the output from your log it works just fine. However the size of data is different in your log from what I see in mine, so I guess that comes from you editing the log.
If you have a valid Kopano subscription, you can open an issue with the support and send me the issue number so that I can look into it. Otherwise you could upload the log file or the relevant part of it and send me the link.
Manfred
-
Hello Manfred,
we have an active subscription. I will compress the log and send it to Kopano itself? Or send it it here to You?
TIA
Uwe Herczeg
-
Hi Uwe,
please send it to Kopano itself (attach the log to the KS issue).
Manfred
-
@Fux and @amaccuish,
the “Line starting at X did not follow iCalendar/vCard conventions” issue on zash1958’s system was caused by a contact with multi line additional information (NOTE property in vcard). Could you please check if you have such contacts and if possible to edit the additional information to be in one line and see if it solves the issue?
I’ve created https://jira.kopano.io/browse/KDAV-28 for this problem.
Manfred