[Solved] Nextcloud, Splunk and Splunk app for Nextcloud


#1

Hi there,
I’ve been trying various ways to be able to read audit the logs from Nextcloud (13.05) I have tried the logviewer but it just doesn’t give enough information. I’ve downloaded the log file and can find the info I’m looking for (user, file accessed, time/date) but I need to be able to submit this to an auditor.

I came across this post Audit and Forensic Logs

Followed the guide and it seemed to work, splunk saw the server and was getting some sort of data (cpu,ram,users,shares) and no errors. However it didn’t seem to be getting the data from nextcloud.log to display all the data graphically on the splunk app for nextcloud.

I imported the file manually from /mnt/ncdata/nextcloud.log and when I searched I could see the data had imported from the log, however it still did not display graphically. I wasn’t sure what format the nextcloud log file was in.

Has anybody else installed splunk and this app and if so is it working? and how did you get it to work correctly?

Or if anybody is willing to install it on their own test bench and troubleshoot with me? as this looks like an incredible app that lots of people are crying out for especially for auditing.

I have a spare Nextcloud instance (14.01) for testing so I have no problem trying anything.

In the meantime I’ll keep testing to see if I can get it to work. I will also post a guide here if I’m successful.

Thanks,

W


#2

Update

So I tried reinstalling Splunk and the Nextcloud app again yesterday evening. I get data but the /mnt/ncdata/nexcloud.log does no seem to be ingested to display through the app. I took a few screen grabs as I progressed to show the various steps I took.

So after installing as per the guide in my earlier post, this is what is displayed.

image
image

It displays that there is on share, I created a test folder.

image

but no data on File and Folder activity

image

It shows data for CPU, RAM and disk space etc

image

but no data on the File, Folder and Audit

image

I added a few users and the new data is detected

image

I added a few files and shared them amongst the new user and data is detected

image

I changed the RAM to 4GB, this change is detected

image

but nothing displaying on Files, Folders and User Audit

image

under search it shows only 31 events

image

so I add the nexcloud.log as per instrustions in earlier post

image

All events are now ingested

image
image
image

But data still not being displayed for these new events on the Splunk Nextcloud app??

Any help would be great.


#3

Is Splunk installed on the same host as Nextcloud or is Splunk installed on a separate host?


#4

It’s installed on the same host, I felt it would be easier that way until I got a full understanding of how it ingests the log.


#5

Assuming Splunk is installed in the default location, log in to your host, then
cd /opt/splunk/etc/apps/TA-nextcloud/local

Edit the inputs.conf file and change the two lines starting with “[monitor”.
Where it says “path/to/”, change that to the real path of the nextcloud.log and the audit.log files.
Right now, for the audit.log file, I’m assuming it looks like this:
[monitor:///path/to/audit.log]

After your change (depending on where the log files are), it might show something like this:
[monitor:///var/www/nextcloud/data/audit.log]

You will need to restart Splunk so it reads the changed configuration file. To restart Splunk, go to "Settings in the Splunk UI, click on “Server Controls” and then it should be obvious what to do.


#6

I did try that but I may have entered the path incorrectly. I removed the /// and typed [monitor:/mnt/ncdata/nextcloud.log]

There are 2 inputs.conf. 1 located in defaults and one in local

and

Do I need to edit both?


#7

You only need to modify the inputs.conf file in one location. Best practice is to modify files in the “local” directory as any updates of the add-on will overwrite the contents of the “default” directory.
As your Nextcloud server is v13, you won’t have an “audit.log” file, so there is no need to modify anything related to it.

Did you only ingest data from “/mnt/ncdata/nexcloud.log” via the upload method, or did you configure Splunk to ingest the file somewhere else? If somewhere else, you will need to disable the ingestion there as Splunk will recognise that it is already ingesting that log file.


#8

I created a checkpoint on my VM before ingesting so I will revert to that point.

My server is 14.03, and I tried to locate the audit.log but I couldn’t find it through root@nextcloud locate -b audit.log ?

I used sudo /opt/splunk/bin/splunk add monitor /mnt/ncdata/nextcloud.log -sourcetype nextcloud-log
to pull the log into splunk, it was there and I could see as the pic above shows but the Splunk app did not process it.


#9

I see, in your first post you wrote that your Nextcloud server is v13. In v14 all the interesting stuff used by the Splunk app is in the “audit.log” file, so you will want to do the same thing you’ve done for the nextcloud.log file with the audit.log file.
If you are doing the “/opt/splunk/bin/splunk add monitor /mnt/ncdata/nextcloud.log -sourcetype nextcloud-log” bit, then modifying the inputs.conf file is not necessary, I may have inadvertently given you wrong directions. Well it can be done either way, but one shouldn’t do it both in the inputs.conf file manually AND by doing the “Splunk add monitor” bit.
Finally, if there is no audit.log file, have you set the loglevel to 1 in the nextcloud configuration file?


#10

One more thing:
If my last message hasn’t solved the issue, In the “Search & Reporting” app, try to enter the following:
“index=* | stats count by source”
and show me via a screenshot what you then see.


#11

I’m sorry, I didn’t realised I’d said 13 … I see now I did … so my production is 13.05 and I used the test to check things on newer versions … current test is 14.03 … my bad

Log level is set to 1 in the config.php and apache restarted

image

using locate audit.log, it only locates the splunk audit.log ??

image

/local/inputs.conf

in data inputs, I followed your earlier steps and edited the /local/inputs.conf and the /mnt/ncdata/nextcloud.log entry appeared and I disabled the /path/to/nextcloud.log and audit.log

search and reporting now shows the events ingested from the log

but no visable change to the visuals in the Nextcloud app

“index=* | stats count by source”


#12

The inputs.conf file in the local directory needs to contain the “sourcetype = nextcloud-log” (without quotes) entry underneath the [monitor… stanza.
In other words, it should look similar to the contents of the inputs.conf file located in the default folder.

About your Nextcloud audit file. Strange, I had a v13 Nextcloud server, and when I upgraded to v14 the audit.log file appeared. The only place I’ve seen it mentioned is in this sentence: “This release introduces a Data Protection Confirmation app and separate audit log file” which can be found here:


The Nextcloud 14 documentation doesn’t mention it, but based on my own experience and that sentence from the release blog post I’m assuming any v14 Nextcloud installation writes to an audit.log file as well as the “old” nextcloud.log file.
My Nextcloud configuration file doesn’t have a file name configured, I’m wondering if by configuring the file name everything IS being written to the nextcloud.log file after all.

Once you’ve modified the inputs.conf file so the sourcetype is defined, and Splunk has been restarted, look at the “Settings > Data Inputs > Files & directories”. In the last screenshot you attached, the “Source type” for your nextcloud.log file is shown as being “Automatic”, it should be “nextcloud-log” like the disabled entries.


#13

I think your problem with the audit.log file not being created is because the Nextcloud “Auditing / Logging” app hasn’t been enabled within Nextcloud.


#14

Yes you’re right, the issue is the with Auditing/Logging app however I did enable it and disable and re-enable but it never created the audit.log? Not one to give up, I came to the realisation that the issue must be with the server and it’s build. I origionally started with 13 and have been updating with each stable release, up to 14.03 so somewhere along the line its stopped functioning as designed.
I noticed some error in the nextcloud.log … redis etc

I decide to go back to the drawing board and rebuild the server with Nextcloud 14.03. Enabled Auditing/Logging and boom audit.log finally appears under /mnt/ncdata

The dashboard immediately lit up and it’s a beauty to behold. You’ve created a beautiful interface.

Your guide is perfect … well during my installation almost perfect :slight_smile:

To add to your guide perhaps, although it may only be in my case but there was no local directory nested in the Splunk directory so I needed to manually create it 1st

so before I ran

sudo -i
cp /opt/splunk/etc/apps/TA_nextcloud/default/TA-nextcloud.conf.sample
/opt/splunk/etc/apps/TA_nextcloud/local/TA-nextcloud.conf
nano /opt/splunk/etc/apps/TA_nextcloud/local/TA-nextcloud.conf

I created a local directory using mkdir local

and to ingest the audit.log, I used

sudo /opt/splunk/bin/splunk add monitor /mnt/ncdata/audit.log -sourcetype nextcloud-log

after all of the step in your guide were complete.

Then restarted splunk.

These steps may have just been applicable to me but I though I’d note it.

So from trial and error I’ve finally got it installed and to show this beautiful dashboard, I’ve added a few pics

I would like to sincerely thank you for your time and help.

To everybody else reading adding this to your Nextcloud server in my opinion brings it up to an Enterprise level installation.