[solved] Corrupted database, needs clean install database

kopanocore-8.6.80.1420 only server, dagent and gateway running

# cat server.cfg | grep -v "^#" | grep -v ^$
server_listen = *:236
local_admin_users = root kopano
mysql_user = kopano
mysql_password = kopano
disabled_features = pop3

exim
fetchmail
evolution-3.28.4 not relinked after updating Gnome, openssl, webkitgtk

Yesterday two emails. one normal readable. The second one has the wrong content when opened in message panel, it shows a deleted email
Test message to myself same: In inbox-panel visible but in message-panel an other deleted email.
Sloved by dropping kopano database and recreating.
This morning 228 emails normal redable, but same problem with two emails from same sender as yesterday
Test email same problem, database corrupted again
This time I have kept the emails on the server of my ISP. On webmail of my ISP I can read both messages
Saved both messages as .eml but I see nothing strange

Return-Path: <SRS0=iulHMwsS=MT=xs4all.nl=goan@srs.dds.nl>
Delivered-To: <tjoen@dds.nl>
Received: from mail2.dds.nl
        by mail2.dds.nl (Dovecot) with LMTP id 6v0jMs5zulutNgEATvFnAA
        for <tjoen@dds.nl>; Sun, 07 Oct 2018 23:02:30 +0200
Received: from relay2.dds.nl (relay2.dds.nl [83.96.176.84])
        by mail2.dds.nl (Postfix) with ESMTP id DDB7B633CC
        for <tjoen@dds.nl>; Sun,  7 Oct 2018 23:02:30 +0200 (CEST)
Received: from mx3.dds.nl (mx3.dds.nl [91.142.252.33])
        by relay2.dds.nl (Postfix) with ESMTP id DC79AC074D7F
        for <tjoen@dds.nl>; Sun,  7 Oct 2018 23:02:30 +0200 (CEST)
Received: from lb3-smtp-cloud7.xs4all.net (lb3-smtp-cloud7.xs4all.net [194.109.24.31])
        by mx3.dds.nl (Postfix) with ESMTPS id 5831D36017C
        for <tjoen@dds.nl>; Sun,  7 Oct 2018 23:02:28 +0200 (CEST)
Received: from Goan01 ([IPv6:2001:984:a103:1:dcb4:16ff:6f33:e213])
        by smtp-cloud7.xs4all.net with ESMTPSA
        id 9GC3gjOxLw2L89GC4gEvO2; Sun, 07 Oct 2018 23:02:28 +0200
Message-ID: <050CD4C3B2AD48BF9676C80B6B4DE2AD@Goan01>
From:  "edited" <edited@edited>
To: "Tjoen Oei" <tjoen@dds.nl>
Subject: Re: Deconstructivisme
Date: Sun, 7 Oct 2018 23:02:25 +0200
MIME-Version: 1.0
Content-Type: text/plain;
        format=flowed;
        charset="iso-8859-1";
        reply-type=response
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
Importance: Normal
X-Mailer: Microsoft Windows Live Mail 15.4.3555.308
X-MimeOLE: Produced By Microsoft MimeOLE V15.4.3555.308
X-CMAE-Envelope: MS4wfCqrw7oaN5x95Q0vUQcsOmPm/h1jujf3sEMIHHAHmP8eXdWlx34SrBJ3Ge3ppTkuGUaPhq+6PvPS09SPfM5cCOJ7lAPdYNdpLFAswAAEBJlL1xQEOq6T
 zujKD9X/Xb2AkbTJciB4V2qNFO0I6zgKKoaKh6xmoDKsLMzLL1EeRcqr7niZclTiw+58xOr4JQw843xK+QxLA9FuOxMWswZNTF8=
X-Virus-Scanned: clamav-milter 0.100.1 at mx3
X-Virus-Status: Clean
X-Spam-Flag: NO
X-Spam-Status: No, score=0.6 required=7.0 tests=AWL=0.192,BAYES_50=0.4,
        RCVD_IN_DNSWL_NONE=-0.0001,RCVD_IN_JMF_W=0.001,SPF_PASS=-0.001 autolearn=no
        version=3.3.1
X-Spam-Report: 
        * -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no
        *      trust
        *      [194.109.24.31 listed in list.dnswl.org]
        *  0.4 BAYES_50 BODY: Bayes spam probability is 40 to 60%
        *      [score: 0.4952]
        * -0.0 SPF_PASS SPF: sender matches SPF record
        *  0.0 RCVD_IN_JMF_W RBL: Sender listed in JMF-WHITE
        *      [194.109.24.31 listed in hostkarma.junkemailfilter.com]
        *  0.2 AWL AWL: Adjusted score from AWL reputation of From: address
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mx3.dds.nl

Schilderij Banksy, ik moest aan jou denken.

Both webmail and Evolution don’t show the first from-line as this example I saved a time ago

From hilmlopiz@hotmail.com Mon May 14 08:01:51 2018
Return-path: <SRS0=Kb0m+ytX=IA=hotmail.com=hilmlopiz@srs.dds.nl>

etc

Strange that email only from one person is causing this problem

Hi @tjoen,

that sounds a bit strange. Some followup questions to this.

@tjoen said in Corrupted database, needs clean install database:

kopanocore-8.6.80.1420 only server, dagent and gateway running

quite a while ago you were asking about running Kopano in LFS is that such a self made build or are you running with our packages (some more recent replies at least suggest you looked at our packages).

@tjoen said in Corrupted database, needs clean install database:

exim
fetchmail
evolution-3.28.4 not relinked after updating Gnome, openssl, webkitgtk

and

@tjoen said in Corrupted database, needs clean install database:

when opened in message panel

just that I fully understand. you are seeing this in an imap client, right? So you are not using Kopano WebApp.

@tjoen said in Corrupted database, needs clean install database:

database corrupted again

if you are indeed using imap, then this may not be a corrupt database after all. For performance reasons since Zarafa 6.40.0 a copy of the full rfc message is stored in the attachment dir. elements in the attachment dir are identified by an “instance id”, which is a simple auto incrementing value. so if you get a wrong item displayed via imap a good test would be if you also get the same wrong item via webapp.

If that isn’t the case (in webapp you see the right object), then something must be wrong with your attachments. this usually only happens if users modify the files in there manually (restoring older backups (attachments do not match highest instance id), restoring backups and having items deduplicated with hardlinks).

@fbartels

quite a while ago you were asking about running Kopano in LFS is that such a self made build or are you running with our packages (some more recent replies at least suggest you looked at our packages).

That was with that MAPI-error. Yes, I have tested in Fedora-28 witch had the same error. Solved by kopanocore-8.6.80.1420

@tjoen said in Corrupted database, needs clean install database:

evolution-3.28.4 not relinked after updating Gnome, openssl, webkitgtk
when opened in message panel

just that I fully understand. you are seeing this in an imap client, right? So you are not using Kopano WebApp.

Evolution as IMAP-client. Most simple configuration is using Kopano as Dovecot replacement, only running server, dagent and gateway. For WebApp more configuration is needed, including kopano-spooler

if you are indeed using imap, then this may not be a corrupt database after all. For performance reasons since Zarafa 6.40.0 a copy of the full rfc message is stored in the attachment dir.

If that isn’t the case (in webapp you see the right object), then something must be wrong with your attachments. this usually only happens if users modify the files in there manually (restoring older backups (attachments do not match highest instance id), restoring backups and having items deduplicated with hardlinks).

That is a good suggestion. I see that the default behaviour is storing attachments.
I’ll delete all attachments before recreating the database. And ask the sender of the problem-email to send a test email.

Deleting attachments before recreating the database was indeed the sollution.