Trying to track down rogue bandwidth usage, logging not working

Hi all, I’m using Nextcloud 10.0.2 hosted on my home internet connection. I’m trying to track down a problem where I’m seeing 2-10 GB downloaded and uploaded every day. I’ve narrowed it down to my Nextcloud server and I’m a bit stuck because none of this data is appearing my normal apache logs. In fact, no file transfers are being logged. That’s fine, perhaps that’s how WebDAV works. Trouble is, they’re not coming up in my data/nextcloud.log either.

I’ve tried changing my config.php from log level 2 to 1 or 0 and restarting apache. Even then, the most recent log entries are stuck at Dec 22 when I upgraded to the latest point release of nextcloud.

It’s possible that the rogue traffic simply a network misconfiguration where my LAN traffic is bouncing off my ISP, but even then I’m surprised by the quantity of traffic and would be much happier if I could see some logs with some IP addresses. Does anyone have any ideas where I can go from here?

The apache-logs should allow you to see the size (you can even use some log analyzing tools.

If apache uses the combined log, you should have something like this:

1.2.3.4 - - [29/Dec/2016:20:27:25 +0000] "GET /nextcloud/index.php/core/preview.png?file=%2Ffolder%2FPhoto-2014-07-29-11-38-29_0389.JPG&c=6a4d2d4823d3b1dc43b64c0741ad9dcb&x=32&y=32&forceIcon=0 HTTP/1.1" 200 1827 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0"

1827 is the number of bytes transferred.

Hi tflidd,

I do see those /core/preview.png logs if I log on via the web interface and open a folder that has images in it. What I don’t see is the sizes of the uploads. I have checked again and now I am successfully seeing the PUT when I upload (not sure what I was doing wrong before). The bytes count on the PUT is always 789 though, regardless of the actual size of the file, which is about 1.6 MB in this case.

192.168.0.50 - user [30/Dec/2016:11:51:02 +0600] "PROPFIND /remote.php/webdav/ HTTP/1.1" 207 15720 "-" "Mozilla/5.0 (Macintosh) mirall/2.2.4 (build 1) (Nextcloud)"
192.168.0.50 - user [30/Dec/2016:11:51:03 +0600] "PUT /remote.php/webdav/IMG_5239.jpg HTTP/1.1" 201 789 "-" "Mozilla/5.0 (Macintosh) mirall/2.2.4 (build 1) (Nextcloud)"

It only shows the size of the sent packages. There is a logging module in apache which exactly does what you want:
https://httpd.apache.org/docs/current/mod/mod_logio.html

Fantastic, that’s exactly what I needed. Thanks! As it turns out that module is compiled in on Debian apache so all I needed to do was define a LogFormat with %I and %O and I’m now seeing all the counts. I’ll leave it running for a while and see if I can understand where gigabytes of data are going.