Z-push / Fail2ban / backend unavailable

sbourdette

Hi,

I have configured fail2ban for z-push to identify user authentification failure.
It’s works well but I have a issue.

Every night, I stop the backend for backup purpose. (30min)
Then every device trying to sync during this time is banned because the authentification to the backend is unavailable.

Is there a way to distinguish wrong credentials and backend timeout to ban only for wrong credentials ?

Regards

Sylvain

~~___________________________________________________________________________________~~
Reply With Quote Reply With Quote Multi-Quote This Message
30-05-2017, 03:42 PM #2
manfred manfred is offline
Senior Member
Join Date
Nov 2007
Location
Belo Horizonte, Brazil
Posts
1,475
Hi Sylvain,

what exactly do mean you stop the backend?

If the authentication is not available, then there’s no way to tell if the credentials are right or wrong.

Manfred

~~__________________________________________________________________________________~~
Reply With Quote Reply With Quote Multi-Quote This Message
30-05-2017, 11:39 PM #3
sbourdette sbourdette is online now
Junior Member
Join Date
Feb 2017
Posts
2
z-push and the zimbra backend are on different servers (VM).
Every night I s"savestate" the Zimbra VM, clone it and then restart it.

During this process Z-push is still receiving sync request but Zimbra is unavailable.
So every sync request fall in timeout but z-push log it as wrong authentification.
I’m asking a RFC to log this case as timeout or unavailable backend instead of wrong authentification.

I hope it’s more clear.

regards

~~______________________________________________________________________________________~~
Reply With Quote Reply With Quote Multi-Quote This Message
31-05-2017, 08:39 PM #4
manfred manfred is offline
Senior Member
Join Date
Nov 2007
Location
Belo Horizonte, Brazil
Posts
1,475
Hi Sylvain,

is it the suggested way to backup Zimbra (I don’t know much about Zimbra)?

You could also configure the Z-Push VM to respond with e.g. 503 (Service Temporarily Unavailable) or some other 500 code. However it might happen that the devices will do a complete resync when they connect again.

Manfred

sbourdette

Hi Manfred,
There are several methods of backup for zimbra.
The issue is independant from this backup.

It’s Z-push that have to change to answer a 503 or another message in the z-push log

Regards

Manfred

@sbourdette said in Z-push / Fail2ban / backend unavailable:

Hi Manfred,
There are several methods of backup for zimbra.
The issue is independant from this backup.

It’s Z-push that have to change to answer a 503 or another message in the z-push log

Regards

Hi Sylvain,

Z-Push doesn’t not know anything about the state of the backend and why it’s not available. Z-Push is only aware that it’s not possible to login. If it’s not possible to login into backend, it will always result into an authentication error.

Z-Push and Zimbra are two different products and Zimbra backend is not a part of the official Z-Push release, so you have to discuss with @liverpoolfcfan if it’s reasonable that Zimbra backend responses with a 503 or a different code in such a case.

Manfred

umgfoin

@sbourdette said in Z-push / Fail2ban / backend unavailable:

Every night, I stop the backend for backup purpose. (30min)
Then every device trying to sync during this time is banned because the authentification to the backend is unavailable.

Hi sbourdette,

assuming your fail2ban jail is called “z-push” and the host serving z-push “zphost”:

Before taking down the backend, call
ssh zphost 'fail2ban-client stop z-push'

When your server is ready to authenticate again, call ssh zphost 'fail2ban-client reload z-push'

The latter will throw an error, which can be ignored - alternatively call reload twice.
The above procedure will prevent fail2ban from reacting on the specific jail while your server is down.

++umgfoin.

liverpoolfcfan

@sbourdette - There are 2 things at play here.

First - your fail2ban rule can be configured to look for x number of failed authentication messages within y seconds for any given IP address.

Second - there is a parameter in the z-push backend config.php file that allows for extending the number of retries to connect to the host before returning a failure to z-push

// On many zimbra FOSS sites the server is stopped each night to facilitate the running of
// a cold sync final backup. This typically means that zimbra can be unavailable for a
// number of minutes. The setting ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR attempts to mitigate
// the potential issues with locking out accounts/dropping data from devices/etc. by 
// holding open login sessions and retrying the authentication a number of times at 60
// second intervals. The default setting allows for 5 minutes of downtime. 
define('ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR',5);

At the default setting of 5 retries it means that the backend will attempt a connection, and upon failure will delay 60 seconds and retry to connect, and repeat this 5 times.

You should be able to tune your ban rule so that it only locks up if it gets 2 or more failures in 290 seconds for example as the loop in the backend should hold the connection for 300 seconds.

You could also try setting this configuration higher, and adjust your fail2ban rule accordingly

Note though that the client may have given up waiting before we return to them and may have re-issued the command, though I haven’t seen this cause a problem in our own environment.

sbourdette

@umgfoin said in Z-push / Fail2ban / backend unavailable:

fail2ban-client stop z-push

Thanks for your answers.
I’m trying to deal with ZIMBRA_RETRIES_ON_HOST_CONNECT_ERROR and fail2ban findtime

Regards

Sylvain

liverpoolfcfan

Did you get it to work for you?

sbourdette

Hi,
I have less ban.

I have to tune the failban parameters. but it’s not the easier way