Kopano Dagent does not deliver incoming Mails // Outbound mails OK

Hello, Community,
I am a little lost now after examining my config again and again - I hope, someone can help me?

With Zarafa, everything used to work fine - but now that I have switched to Kopano, only outbound Mails are delivered.
Not a single incoming Mail successful so far :-(
My Setup:
Copano Core + Copano WebApp + Postfix + Fetchmail
WebApp is reachable, I can login to both of the configured users.
I can create new Mails and send them externally.
The sent mails are correctly stored in the sent folder.

What does NOT work:

I cannot send a simple Mail from user1@domain.local --> user2@domain.local
Postfix always uses my relaymap and accordingly gets error messages from GMX, that the recipient’s adress is invalid. Thats true - but I didn’t want to deliver the .local Mail to GMX!
What did I do wrong? The relaymap itself works, otherwise I wouldnt be able to send mails out?
No Incoming Mails at all! In Zarafa I used fetchmail to pull the Mails from GMX. But this seems no longer to work - fetchmail successfully receives the Mails(as per log file), but - no idea where the mails are?? At least they are NOT delivered to the local user1 or user2
Any idea, where fetchmail has put my mails to?

I have anonymized the output below, but hope the general setup is clear.

Thanks for ANY hint on how to get my mails back or at least the new ones delivered!

Please help, I have no idea anymore.

P.S. I saw a hint here in the forum about a bug in dagent, but the mentioned errors do not appear in my kopano-dagent.log! So I assume, I am not affected? Or not verbose enough?

Hoinz

/etc/postfix/main.cf

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = domain.local
virtual_alias_maps = hash:/etc/postfix/virtual
virtual_mailbox_maps = hash:/etc/postfix/virtual
virtual_transport = lmtp:127.0.0.1:2003
virtual_mailbox_domains = example.com, example.org, example.net
myorigin = /etc/mailname
mydestination = localdomain.tld, localhost
relayhost =
mynetworks = 192.168.2.0/24 127.0.0.0/8
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
html_directory = /usr/share/doc/postfix/html
#Outbound Relay
sender_dependent_relayhost_maps =hash:/etc/postfix/relaymap
smtp_sender_dependent_authentication = yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/passes
smtp_sasl_security_options = noanonymous

/etc/fetchmailrc

set daemon 120
poll ******.de protocol pop3 interval 1 user "*******@*****.de" password "*********" is user1 here ssl

poll pop.gmx.net protocol pop3 interval 1 user "********.*****@gmx.de" password "***********" is user2 here ssl

set logfile = /var/log/fetchmail.log
set no syslog

/etc/kopano/dagent.cfg

##############################################################
# DAGENT SETTINGS

# connection to the storage server
server_socket = file:///var/run/kopano/server.sock

##############################################################
# DAGENT SSL LOGIN SETTINGS
#
# Note: server_socket must be set to https://servername:portname/
#       to use this type of login method

# Login to the storage server using this SSL Key
#sslkey_file = /etc/kopano/ssl/dagent.pem

# The password of the SSL Key
#sslkey_pass = replace-with-dagent-cert-password

##############################################################
# DAGENT LOG SETTINGS

# Logging method (syslog, file)
log_method      =       file

# Loglevel (0(none), 1(crit), 2(err), 3(warn), 4(notice), 5(info), 6(debug))
log_level       =       6

# Logfile for log_method = file, use '-' for stderr
log_file = /var/log/kopano/dagent.log

# Log timestamp - prefix each log line with timestamp in 'file' logging mode
log_timestamp   =       1

Log-Files: When sending Mails, all OK
For receiving Mails – I just see fetchmail getting the Mail
/var/log/mail.log does not show any incoming mail activity.

fbartels

Hi @hoinz_p ,

normally you should be able to reuse your old zarafa mta configuration as is, as all the mechanics stayed the same.

You are making a bit contradicting statements, at first you say that the mta tries to relay your local mails to your smarthost, then you say there is no activity in the mail.log upon delivery. From the first statement I would guess that you are missing your local domain in the virtual_mailbox_domains.

Hello, Felix,
thanks for your answer.
With “no mail activity” I meant incoming mail towards the local user.

For Example, this is the log file for an outgoing, successful Mail delivery:

Apr 22 22:22:04 plett-hub postfix/smtp[27896]: BED9512005D: to=<********.#####@gmx.de>, relay=mail.hplett.de[83.169.16.162]:25, delay=0.73, delays=0.09/0.03/0.43/0.18, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 52BC480734)
Apr 22 22:22:04 plett-hub postfix/qmgr[27887]: BED9512005D: removed

And here are the Mails that are obviously come in (after I switched on fetchmail, to not lose mail, I switch it off normally)

Apr 22 22:29:16 plett-hub postfix/cleanup[28141]: 01A5812005D: message-id=<003101d2b833$5bd3b650$137b22f0$@toews.de>
Apr 22 22:29:16 plett-hub postfix/qmgr[28052]: 01A5812005D: from=<*******@toews.de>, size=679984, nrcpt=1 (queue active)
Apr 22 22:29:16 plett-hub postfix/trivial-rewrite[28140]: warning: do not list domain localhost in BOTH mydestination and virtual_mailbox_domains
Apr 22 22:29:16 plett-hub postfix/local[28142]: 01A5812005D: to=<heinrich@localhost>, relay=local, delay=0.49, delays=0.43/0/0/0.06, dsn=2.0.0, status=sent (delivered to mailbox)

OK, you are right, this is incoming Mail, but I don’t know, where the mails are stored, as they do NOT appear in the WebApp.

What exactly means “Mailbox”?

(delivered to mailbox)

To explain the log output: My hostname is “plett-hub” and my user 1 is “heinrich”
So, for the output above I again “lost” mail :-( despite I hope that they actually are stored “somewhere”.

Any other idea?

I need to add, that when I run “mutt” from the shell, I find the mails that fetchmail has pulled from the server. Bingo!

The Mail store for Mutt is /var/mail/heinrich

How can I make it to deliver them to the Kopano Web App?

How can get my Mails into WebApp instead of mutt?

There is only 1 user “heinrich” on the system - so why is the eMail delivered to mutt, but not to copano?
Also the kopano user “heinrich” has the same password like the system user “heinrich” - so I would expect that the delivery should work properly.

Any other suggestions?

fbartels

Kopano

Is it really heinrich@localhost? It seems that you lookups in virtual_alias_maps and virtual_mailbox_maps are not successful, and therefore he defaults back to local delivery.

Hi All,
I have tried now literally ANY how-to, but never succeed.
The original Kopano documentation may be good, but it confuses me with all of the different options(DB-plugin,LDAP, virtual), but I have not seen a complete main.cf.
@Felix, I guess you are right, that it might be somehow related to my bad local user lookup - anyway, I changed this also to the mysql version (mysql:/etc/postfix/mysql-users.cf), but also this doesn’t work.
Is there a chance that someone uploads a working main.cf with basic functionality?

fbartels

@hoinz_p you are using users in MySQL?

Hi Felix,
yes that is the setup as described in the Kopano Documentation:

alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
virtual_alias_maps = mysql:/etc/postfix/mysql-users.cf

Where “mysql-users.cf” contains an Mysql request:

user = kopano
password = *****
hosts = 127.0.0.1
dbname = kopano
query = select value from objectproperty where \
        objectid=(select objectid from objectproperty where value='%s' limit 1) \
        and propname='loginname';

That was just one more attempt of configuration.
You said its the same like Zarafa - definitively NOT :-) In Zarafa I worked with “real” users as they exist on my Ubuntu system.
Here it seems that its no longer possible(only LDAP, AD,Mysql or Virtual).
As I don’t want to run into more problems, I don’t try LDAP and AD, but just tried the 2 things: Virtual Users and MySQL.

In the meantime (I did more testing yesterday night) I managed to send and receive mails within the Mail application “mutt”. When I had Zarafa, the incoming Mails were visible in Mutt and Zarafa Webaccess in parallel.
This is also different here. I see the mails in Mutt, but I do not see them in Kopano WebApp.
The WebApp seems to work in General(as I get “not delivered” replies), but it seems to have the mails stored in a different location.
Can you tell me, how to influence it?
Can I tell Postfix, where to store the Mails?

Thanks for your time!

Hoinz

fbartels

Hello @hoinz_p ,

I still have to find the time to write a more verbose reply, but already wanted to comment on the following.

@hoinz_p said in Kopano Dagent does not deliver incoming Mails // Outbound mails OK:

You said its the same like Zarafa - definitively NOT :-) In Zarafa I worked with “real” users as they exist on my Ubuntu system.
Here it seems that its no longer possible(only LDAP, AD,Mysql or Virtual).

the unix backend, while surely not actively tested for a while, is still part of the codebase and can be used. https://stash.kopano.io/projects/KC/repos/kopanocore/browse/installer/linux/server.cfg#335-338

Dear Felix,
still urgently need your support.

Progress so far: I found a mistake in my relaymap, so that the delivery agent did not even kick in.
This is now solved, I see the delivery agent working.
However, still the Mails are not delivered into the mailbox.

I have 2 users, ‘heinrich’ and ‘gmx’

My fetchmail config says

poll xxxxxxx.de protocol pop3 interval 1 user "heinrichxxxx@xxxxxxx.de" password "xxxxxxx" is heinrich here ssl

poll pop.gmx.net protocol pop3 interval 1 user "heinrichxxxx@gmx.de" password "xxxxxxxxx" is gmx here ssl

The above config works in first instance, however, the delivery agent does not resolve “heinrich” or “gmx” correctly. Normally he should just deliver to the system users ‘heinrich’ and ‘gmx’. But it doesnt:

Thu May  4 22:42:01 2017: [debug  ] [ 7800] < 250-SERVER ready
Thu May  4 22:42:01 2017: [debug  ] [ 7800] < 250-PIPELINING
Thu May  4 22:42:01 2017: [debug  ] [ 7800] < 250-ENHANCEDSTATUSCODE
Thu May  4 22:42:01 2017: [debug  ] [ 7800] < 250 RSET
Thu May  4 22:42:01 2017: [debug  ] [ 7800] > MAIL FROM:<heinrichxxxx@gmx.de>
Thu May  4 22:42:01 2017: [debug  ] [ 7800] < 250 2.1.0 Ok
Thu May  4 22:42:01 2017: [debug  ] [ 7800] > RCPT TO:<heinrich@localhost>
Thu May  4 22:42:01 2017: [debug  ] [ 7800] Resolved command "RCPT TO:<heinrich@localhost>" to recipient address "heinrich@localhost"
Thu May  4 22:42:01 2017: [error  ] [ 7800] Failed to resolve recipient heinrich@localhost (0)
Thu May  4 22:42:01 2017: [error  ] [ 7800] Requested e-mail address 'heinrich@localhost' does not resolve to a user.

So, any idea, where does the @localhost come from ???

My latest main.cf is

mydomain = xxxx-hub.local
mydestination = xxxx-hub.local, localhost

smtpd_banner = $myhostname ESMTP
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = /usr/share/doc/postfix

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtp_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.
myhostname = plett-hub
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
virtual_alias_domains = xxxxxx.de, gmx.de
virtual_alias_maps = hash:/etc/postfix/virtual
mailbox_transport = lmtp:localhost:2003
mynetworks = 192.168.2.0/24 10.124.205.0/24 10.88.36.0/24 10.88.37.0/24 127.0.0.0/8 [::1]/128
mailbox_size_limit = 51200000
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
#Outbound Relay
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
sender_dependent_relayhost_maps =hash:/etc/postfix/relaymap
smtp_sender_dependent_authentication = yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/passes
smtp_sasl_security_options = noanonymous

And the Virtual alias file is

heinrichxxxx@xxxxxx.de              heinrichxxxx@xxxxxx.de ### so, just the real eMail addresses
heinrichxxxx@gmx.de          heinrichxxxx@gmx.de

Thanks!

Heinrich

Oops, just realized, Felix seems to be on holiday …:boy_tone2:
Anybody else out there with an idea, please?

The Question is, why my ‘username’ gets translated into ‘username@localhost’, despite I have not configured this?
And WHICH application is responsible? Fetchmail? Postfix? Kopano? See also the post above.

Thanks!

Hi,
is there anybody to reply while fbartels is on leave?

Still my problem is, that I cannot get Kopano working to receive eMails.
Fetchmail seems not capable to deliver to virtual Mailboxes.
Kopano on the other hand seems not to work as zarafa used to (with system users).
So, what is the recommended way from Kopano? How can inbound Mails be received?

In the Manual, I always find documentation about OUTbound mail (via postfix + dagent + spooler) but I have not seen any hint about INcoming mails.

Can anybody help?

umgfoin

You’re configuring virtual domains, but setting up mailbox_transport.
virtual_mailbox_transportmight be your intention.

Useful information on Postfix virtual domains: VIRTUAL_README

Is the kopano-user configured with both email-adresses of heinrich (heinrichxxxx@xxxxxx.de and heinrichxxxx@gmx.de)?

++umgfoin.

.

Hello, umgfoin,

thanks for replying.

You are right, I am mixing this up and this reflects my situation - in Zarafa, I never had to deal with virtual users.
So, am I right, that fetchmail is unable to handle delivery to virtual boxes?

Users: I have 2 users: ‘heinrich’ is one, ‘gmx’ the other. Each of them has only one eMail address.

Does this inspire you for more ideas? ;-)

Thanks!

Heinrich

Hello,

has anybody more hints for me?

Current status:
Outbound Mail works.
Inbound mail to gmx works - because I use an alias file.
Surprisingly, this does not work for the other user.
Postfix returns a “mail loop” notification - despite the fact, that I do absolutely the same like for the gmx user.
So, is Kopano limited to 1 user only?

How do you guys deal with inbound mails?
Any other suggestions besides Fetchmail?

I always read descriptions for outgoing mail - but what is a working solution for inbound mail ??

Thanks!

Heinrich

robgnu

Hello,
I hope, I can help. Why do you forward incoming mails from fetchmail to postfix? I think it is more elegant to send them directly to the kopano-dagent. I do this on some setups as well.

Please give it a try with this fetchmail configuration:

poll example.com with proto POP3
        user "heinrich@****.de" password "****"
        options ssl sslcertck
        mda "/usr/sbin/kopano-dagent heinrich"

poll pop.gmx.net with proto POP3
        user "*****@gmx.de" password "****"
        options ssl sslcertck
        mda "/usr/sbin/kopano-dagent gmx"

Please also check whether your kopano-dagent is enabled. On Debian/Ubuntu it is one line in /etc/default/kopano:
DAGENT_ENABLED=yes

After that you can optinally use transport maps for outgoing mails. When sending mails to local users it is more elegant to not send them out to your smarthost and fetch them with fetchmail. Just create a postfix lookup file like this. If the recipient is a kopano user, postfix delivers the mail directly to the kopano-dagent instead of sending the mail to your smarthost.

/etc/postfix/transport.mysql:

user = dbuser
password = dbpass
hosts = 127.0.0.1:3306
dbname = kopano
query = select 'lmtp:localhost:2003' from objectproperty where objectid=(select objectid from objectproperty where value='%s' limit 1) and propname='loginname';

Enable this file in your main.cf:

transport_maps = mysql:/etc/postfix/transport.mysql

I hope it helps and I would be happy to know if it has worked.

best regards,
Rob

Hello, Rob,

thanks for your idea. Unfortunately, this doesn’t work either.
I reconfigured my fetchmailrc like you suggested, but then just nothing happened, no log entry in the mail.log.
However, dagent.log returned the following:

Unable to open logfile '/var/log/kopano/dagent.log' as user 'kopano'
Not enough permissions to append logfile '/var/log/kopano/dagent.log'. Reverting to stderr.
Sat May 27 23:21:52 2017: [31652] [error  ]   Python type: (null)
Sat May 27 23:21:52 2017: [31652] [error  ]   Python error: 'module' object has no attribute 'DAgentPluginmanager'
Sat May 27 23:21:52 2017: [31652] [crit   ] K-1732: Unable to initialize the dagent plugin manager: Unknown error code (1).

So, obviously, the plugin doesn’t work.

Is this a general Kopano issue (as I saw some tread about this somewhere here in the Forum) ?

In General, the dagent was already set to “yes”, so that was working before.

Once I have the inbound mails working, I might try the transport maps as you suggested …

Any other idea for inbound mail delivery please? Which other options do you know besides fetchmail?

Thank you so much for taking care!

Heinrich

OK, I followed that tread: link text and disabled the plugin manager. Now I receive Mails on both accounts! WOOW ! ;-) Do you think this is a sustainable solution? What else will not work, while the plugin Manager is disabled? Also do you have an idea, why the log says Unable to open logfile '/var/log/kopano/dagent.log' as user 'kopano' – when I issue an ls -la, then I see ```
ls -la /var/log/kopano/dagent.log
-rw-r–r-- 1 kopano kopano 78642 Mai 27 23:54 /var/log/kopano/dagent.log

Hmm I am afraid I was happy TOO early … now everything is slow, from the “heinrich” account I hardly can do anything - not even contacts can be resolved.
:-( :-( :-(