Fail2ban Nextclouds log expression chaged

Since Nextcloud 16 the log expression changed, therefore my fail2ban isn’t catching the log expression, do you know how to change?

here my fail2ban regex

failregex = ^{"reqId":".*","remoteAddr":".*","app":"core","message":"Login failed: '.*' \(Remote IP: '<HOST>'\)","level":2,"time":".*"}$
        ^{"reqId":".*","level":2,"time":".*","remoteAddr":".*","app":"core".*","message":"Login failed: '.*' \(Remote IP: '<HOST>'\)".*}$

ignoreregex =

nextcloud.log:

{"reqId":"XXGK5nx0-xjhgkl","level":2,"time":"September 05, 2019 22:23:36","remoteAddr":"IPHIDDEN","user":"--","app":"core","method":"POST","url":"\/index.php\/login?redirect_url=\/index.php\/apps\/news\/&user=asdf","message":"Login failed: 'test' (Remote IP: 'IPHIDDEN')","userAgent":"Mozilla\/5.0 (rv:68.0) Gecko\/20100101 Firefox\/68.0","version":"16.0.4.1"}

1 Like

Could you indent the regex lines by 4 spaces so it shows up in monospace font? Regexes are difficult to parse on their own, and the Discourse parser is showing your text wrapped in asterisks as italics. It’s also clobbering your quotes.

I took the liberty to attempt fixing it:

With that out of the way, fail2ban really only needs a message to identify an attempt and the IP address to ban. We can simplify your regex with this knowledge. So you’ll need only the message — though we should grab the remoteAddr because it’s easier to get to than complicating our message part — and probably add core so we know it’s Nextcloud and not some other service that a Nextcloud app might login to.

fail2ban uses Python regexes, and has a few extensions to make life easier. My NC instance isn’t under attack so I can’t test this, but I’ve added it to my f2b config in case it ever is.

It includes a <HOST> directive to match IP addresses, i.e. what’s in remoteAddr. So try this on for size:

failregex = ^{.*"remoteAddr":"<HOST>".*"app":"core".*"message":"Login failed.*".*}$

This regex was crafted at regex101 and I converted it to something fail2ban might accept. Here’s the original PCRE that I matched with:

^\{.*\"remoteAddr\":\"(.*?)\".*\"app\":\"core\".*\"message\":\"Login failed.*\".*\}$

See it live

You can test the regex on your logs with fail2ban-regex. The first argument is a log line to match against, and the second is the regex to match with. You can also hook it up to your logs directly, but consult --help for that. So…

fail2ban-regex '{"reqId":"XXGK5nx0-xjhgkl","level":2,"time":"September 05, 2019 22:23:36","remoteAddr":"127.0.0.1","user":"–","app":"core","method":"POST","url":"/index.php/login?redirect_url=/index.php/apps/news/&user=asdf","message":"Login failed: ‘test’ (Remote IP: ‘IPHIDDEN’)","userAgent":"Mozilla/5.0 (rv:68.0) Gecko/20100101 Firefox/68.0","version":"16.0.4.1"}' '^{.*"remoteAddr":"<HOST>".*"app":"core".*"message":"Login failed.*".*}$'

Running that should show a match. If it doesn’t, then fail2ban needs to be taught how to find Nextcloud’s timestamp, or Nextcloud’s logging timestamp format should be changed to something that fail2ban recognizes. fail2ban won’t match any log line until it can find a timestamp first.

Your timestamp format doesn’t appear to be recognized by fail2ban, explaining why it isn’t triggering your actions.

For what it’s worth, my Nextcloud (15.0.10) log has timestamps that look like 2019-09-07T03:35:35+00:00, which is known as ISO 8601. The Nextcloud documentation on Logging covers a config.php option that allows you to set the timestamp format:

/* config.php */
'logdateformat' => "c"

As seen in the PHP date() docs, the c expands to a full ISO 8601 timestamp for you, which is practically universal and also supported by fail2ban. Setting this will also ensure that future updates to Nextcloud won’t break your logging; at least not the timestamp part! :stuck_out_tongue:

I hope this helps.

3 Likes

Well!

Thank you for the hint with the timestamps! Very helpful and good to know !!!

Hey, maybe you can me: I’ve set everything and fail2ban-regex gives following:

user@cloud:~$ fail2ban-regex /var/log/nextcloud.log /etc/fail2ban/filter.d/nextcloud.conf

Running tests
=============

Use   failregex filter file : nextcloud, basedir: /etc/fail2ban
Use         log file : /var/log/nextcloud.log
Use         encoding : UTF-8


Results
=======

Failregex: 16 total
|-  #) [# of hits] regular expression
|   1) [16] ^{.*"remoteAddr":"<HOST>".*"app":"core".*"message":"Login failed.*".*}$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [124] ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T|  ?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
`-

As I see there are 16 hits and a date template is also detected. The jail is active but I’ll not be banned…

/etc/fail2ban/filter.d/nextcloud.conf

[Definition]
failregex = ^{.*"remoteAddr":"<HOST>".*"app":"core".*"message":"Login failed.*".*}$
ignoreregex =

/etc/fail2ban/jail.local

[nextcloud]
enabled = true
port = 80,443
protocol = tcp
filter = nextcloud
maxretry = 2
logpath = /var/log/nextcloud.log

(set it on “2” for testing)

The status:

user@cloud:~$ sudo fail2ban-client status nextcloud
Status for the jail: nextcloud
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     0
|  `- File list:        /var/log/nextcloud.log
`- Actions
|- Currently banned: 0
|- Total banned:     0
`- Banned IP list:

And an example string:

{"reqId":"iOZilFr4RiAdHzxaTyP7","level":2,"time":"2019-09-12T12:09:14+02:00","remoteAddr":"***.***.***.***","user":"--","app":"core","method":"POST","url":"\/login?redirect_url=\/settings","message":"Login failed: 'TheEvil' (Remote IP: '***.***.***.***')","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko\/20100101 Firefox\/69.0","version":"16.0.4.1"}

What did I do wrong?

Can you search your fail2ban.log for messages regarding that filter?
Do you see some “[nextcloud] Found …” in the fail2ban.log?

Did you configure “ignoreip” in the jail.conf/ jail.local?

How about this one? Simple and still works with 16.

^{.*Login failed: '.*' \(Remote IP: '<HOST>'\).*}$

Thanks for your effort, I have just localhost ignored. Actually I didn’t looked up if fail2ban creates a log file :roll_eyes:

I am sure that I reloaded and restarted the service multiple times after making changes. Now I cleared the nextcloud- and the fail2ban-log (no “found” entries) and tried it again and it works. I feel like a noob! :fearful:

Don’t :slight_smile:
That’s IT, not everything can be explained and sometimes weird things happen :smiley: