Fail2ban Native Support

MichaIng · March 28, 2017, 9:18am

Hmm, since there is brute force protection baked into nextcloud, is there any advantage using fail2ban for nextcloud login on top of that?

hailthemelody · March 28, 2017, 1:22pm

The guide I created was made during a time with brute-force protection was not available in Nextcloud. I will update my guide to mention that it’s now an available option.

I have not experimented with the difference between the two (or other) solutions, though for forward-compatibility it’s probably more wise to go with Nextcloud’s implementation.

MichaIng · March 28, 2017, 3:15pm

As far as I know, the nextcloud brute force protection leads to a growing delay between failing login attempts. Fail2ban on the other hand completely blocks a certain ip, if it fails given times for a given duration. But fail2ban is configurable in allowed attempts and block duration. So theoretically both together lead to a little stricter blocking/delaying rules and work fine together (in my case). I don’t know about performance/memory usage etc. but I guess watching one log file (fail2ban) more should be not noticeable and it is anyway highly recommended for SSH.

But yeah, doubles the thing little bit now .

riegerCLOUD · March 29, 2017, 1:13pm

doubles ? yes. nevertheless: for security reasons and from my perspective

https://www.c-rieger.de/nextcloud-installation-guide/#c103
it is higly recommended!
With an effort less than 5 minutes your system is more secure…it is up to you
cheers, carsten

jksinton · March 30, 2017, 2:05am

Fail2ban blocks the untrusted IP at the interface level using a smart firewall. Once blocked via fail2ban, the untrusted IP never accesses your HTTP server, whereas the throttling built-in to NC is done at the PHP level, and thus, allows the untrusted user to access server resources, such as Apache, mysql, PHP, etc. I use both forms of brute force protection. It’s always better to have redundancy especially for brute force.

riegerCLOUD · March 30, 2017, 4:29am

and you won’t mention any performance impact

jakobssystems · March 30, 2017, 5:38am

as alternative I am using DenyHosts… basically same

Auguss · April 1, 2020, 2:00am

Currently using Nextcloud 18. I have tried the suggestions on this site, went to a regex creator on a website and manually typed the regex filter creator/tester website character by character to cover the output syntax, followed the formatting of start filter and end filter displayed. I cannot the fail2ban-regex to recognize the logged output. The filters I have tried are(copied from fail2ban-regex output): postimg.cc/4KZLyQWm

I am unable to hit any of the logging output

Paradox551 · April 1, 2020, 2:06am

The regex for fail2ban changes on basically every major update.

I would check out nextcloudpi and see what they use in their regex.

Bernie_O · April 1, 2020, 5:50am

I use a more generic regex which works fine for Nextcloud 12 through 18:

[Definition]
# regex for Nextcloud 12 - 18:
failregex = ^.*Login failed: '?.*'? \(Remote IP: '?<HOST>'?\).*$

I am quite confident that this will serve my needs also with future releases of Nextcloud.

gas85 · April 1, 2020, 8:49am

Exactly this will not work always anymore, please see discussion under:

also in provided GIST.

Bernie_O · April 1, 2020, 1:08pm

Oh, could you elaborate in which cases this will not work always anymore? I checked your link and could not find a reason. It works with my installation (18.0.3) and I don‘t understand why it shouldn’t work with other installations as well since I didn‘t change the nextcloud source code.

gas85 · April 1, 2020, 2:21pm

I did this search in the NC 17 https://github.com/nextcloud/server/search?q="(Remote+IP%3A"&type=Code and at this time it shows that it was used differently. I could not find it now quickly
But you can see from Gist that Webdav did generate output with IP in ’ ’ and WebUI without.

Bernie_O · April 1, 2020, 2:22pm

My regex catches both.

gas85 · April 2, 2020, 7:55am

You right, I did not read it correctly first time. Thanks! Will go update my gist also

Auguss · April 3, 2020, 1:56am

I have tried your filter. It does not work for NC18. I dont understand what is wrong with the regex. It works on the regex website testers. But will not work on Fail2Ban-regex

gas85 · April 3, 2020, 7:52am

Just tested it on NC 18.0.3 and it works. Set as described in attached GIST:

# fail2ban-client status nextcloud
Status for the jail: nextcloud
|- Filter
|  |- Currently failed:	0
|  |- Total failed:	3
|  `- File list:	/var/nextcloud/data/nextcloud.log
`- Actions
   |- Currently banned:	1
   |- Total banned:	1
   `- Banned IP list:	10.10.10.10
# fail2ban-regex /var/log/nextcloud.log /etc/fail2ban/filter.d/nextcloud.conf --print-all-matched

Running tests
=============

Use   failregex filter file : nextcloud, basedir: /etc/fail2ban
Use         log file : /var/log/nextcloud.log
Use         encoding : UTF-8


Results
=======

Failregex: 8 total
|-  #) [# of hits] regular expression
|   1) [8] ^.*Login failed: '?.*'? \(Remote IP: '?<HOST>'?\).*$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1521] ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|  ?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
`-

Lines: 1521 lines, 0 ignored, 8 matched, 1513 missed
[processed in 0.73 sec]

|- Matched line(s):
|  {"reqId":"6lp8QUUB6ijmdR0YOL9E","level":2,"time":"2020-03-21T12:27:11+00:00","remoteAddr":"xxx.xxx.xxx.xxx","user":"--","app":"core","method":"GET","url":"\/favicon.ico","message":"Login failed: 'user' (Remote IP: 'xxx.xxx.xxx.xxx')","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko\/20100101 Firefox\/74.0","version":"17.0.4.2"}
|  {"reqId":"s1k3BNjbU9sK26jX2tk9","level":2,"time":"2020-03-21T12:27:11+00:00","remoteAddr":"xxx.xxx.xxx.xxx","user":"--","app":"core","method":"GET","url":"\/index.php\/login","message":"Login failed: 'user' (Remote IP: 'xxx.xxx.xxx.xxx')","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko\/20100101 Firefox\/74.0","version":"17.0.4.2"}
|  {"reqId":"ARHJI0F2cZF2wnClmra6","level":2,"time":"2020-03-21T12:27:42+00:00","remoteAddr":"xxx.xxx.xxx.xxx","user":"--","app":"core","method":"GET","url":"\/favicon.ico","message":"Login failed: 'user' (Remote IP: 'xxx.xxx.xxx.xxx')","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko\/20100101 Firefox\/74.0","version":"17.0.4.2"}
|  {"reqId":"OCTSEPCdSr5py5NEzJcm","level":2,"time":"2020-03-21T12:27:43+00:00","remoteAddr":"xxx.xxx.xxx.xxx","user":"--","app":"core","method":"GET","url":"\/index.php\/login","message":"Login failed: 'user' (Remote IP: 'xxx.xxx.xxx.xxx')","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko\/20100101 Firefox\/74.0","version":"17.0.4.2"}
|  {"reqId":"dTLpX1auaSAWvp4xr8n8","level":2,"time":"2020-04-03T07:44:47+00:00","remoteAddr":"10.10.10.10","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: test (Remote IP: 10.10.10.10)","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0","version":"18.0.3.0"}
|  {"reqId":"BAq9dCpbiG5Hml0NuHrd","level":2,"time":"2020-04-03T07:49:15+00:00","remoteAddr":"10.10.10.10","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: test1 (Remote IP: 10.10.10.10)","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0","version":"18.0.3.0"}
|  {"reqId":"yr5SPAeXW0q4tNX2yUOK","level":2,"time":"2020-04-03T07:49:22+00:00","remoteAddr":"10.10.10.10","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: test2 (Remote IP: 10.10.10.10)","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0","version":"18.0.3.0"}
|  {"reqId":"JGkhwlcOLyc2Zwg2u2GT","level":2,"time":"2020-04-03T07:49:26+00:00","remoteAddr":"10.10.10.10","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: test3 (Remote IP: 10.10.10.10)","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0","version":"18.0.3.0"}
`-
Missed line(s): too many to print.  Use --print-all-missed to print all 1513 lines

Auguss · April 3, 2020, 5:28pm

root@DVR:/home/(omitted)# fail2ban-regex -v --print-all-missed /var/log/nextcloud/nextcloud.log /etc/fail2ban/filter.d/nextcloud.conf | grep "Login failed"
|   1) [0] ^.*Login failed: '?.*'? \(Remote IP: '?<HOST>'?\).*$
|  {"reqId":"32hK455sg5j4G9N7lxkd","level":2,"time":"March 31, 2020 12:47:40","remoteAddr":"127.0.0.1","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: (Omitted)(Remote IP: 123.5.5.5)","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0","version":"18.0.3.0"}
|  {"reqId":"nLlrDeGZAbbhuNNgIzum","level":2,"time":"March 31, 2020 19:58:25","remoteAddr":"162.210.195.157","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: (Omitted) (Remote IP: 162.210.195.157)","userAgent":"Mozilla/5.0 (Linux; Android 8.0.0; LG-LS993) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Mobile Safari/537.36","version":"18.0.3.0"}
|  {"reqId":"Ai3vmpN9xExg9j5aGaqh","level":2,"time":"April 01, 2020 16:21:59","remoteAddr":"127.0.0.1","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: (Omitted) (Remote IP: 127.0.0.1)","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:74.0) Gecko/20100101 Firefox/74.0","version":"18.0.3.0"}
|  {"reqId":"ubd2IVWu2eNFntdLrj02","level":2,"time":"April 02, 2020 21:38:10","remoteAddr":"23.82.14.14","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: Test (Remote IP: 23.82.14.14)","userAgent":"Mozilla/5.0 (Linux; Android 8.0.0; LG-LS993) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Mobile Safari/537.36","version":"18.0.3.0"}`

What am I doing wrong? Is date template hits a requirement?

Bernie_O · April 4, 2020, 3:35pm

As far as I know fail2ban is not able to interpret this time stamp. Can you either delete logdateformat from Nextclouds config.php or change the value to:

logdateformat => 'c',

which makes PHP to expand the timestamp to ISO8601. fail2ban is able to interpret ISO8601 time stamps.

Also make sure that the value of logtimezone in Nextclouds config.php corresponds to the timezone of the system (cat /etc/timezone)

Auguss · April 9, 2020, 12:48am

2020-04-08 20:40:22,413 fail2ban.filter         [11616]: WARNING Found a match for '{"reqId":"3TI2KWE6sVyHK4A5GPmm","level":2,"time":"2020-04-08T20:40:21-04:00","remoteAddr":"23.82.14.36","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: Test (Remote IP: 23.82.14.36)","userAgent":"Mozilla/5.0 (Linux; Android 8.0.0; LG-LS993) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Mobile Safari/537.36","version":"18.0.3.0"}' but no valid date/time found for '{"reqId":"3TI2KWE6sVyHK4A5GPmm","level":2,"time":"2020-04-08T20:40:21-04:00","remoteAddr":"23.82.14.36","user":"--","app":"no app in context","method":"POST","url":"/index.php/login","message":"Login failed: Test (Remote IP: 23.82.14.36)","userAgent":"Mozilla/5.0 (Linux; Android 8.0.0; LG-LS993) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Mobile Safari/537.36","version":"18.0.3.0"}'. Please try setting a custom date pattern (see man page jail.conf(5)). If format is complex, please file a detailed issue on https://github.com/fail2ban/fail2ban/issues in order to get support for this format

Failregex: 0 total
|-  #) [# of hits] regular expression
|   1) [0] ^.*Login failed: '?.*'? \(Remote IP: '?<HOST>'?\).*$
|   2) [0] ^.*\"remoteAddr\":\"<HOST>\".*Trusted domain error.*\$


Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [0] ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|  ?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?