When I download multiple files at once the downloads randomly stop

Nextcloud version:
16.0.1

Operating system and version:
Ubuntu 18.04.2 LTS

Apache or nginx version:
Apache/2.4.29 (Ubuntu)

PHP version:
PHP 7.2.17-0ubuntu0.18.04.1

The issue you are facing:
When I download many files from my nextcloud server, the download stops after a seemingly random amount of seconds (between 1 and ~100 seconds).
I don’t have issues downloading a single large file with for example 32GB. I also have no problems uploading any amount of files to my nextcloud server via a webbrowser or the windows client. The issue is solely isolated on downloading multiple files at once.

Is this the first time you’ve seen this error? :
Yes, but I didn’t test it before. I mainly synced TO my server - not FROM it.

Steps to replicate it:
-Via Browser:

  1. open the url of your nextcloud server
  2. log in
  3. Click on the three dots next to a folder with multiple files in it (for example > 100 files)
  4. Hit the download button

-Via Windows Client:

  1. Upload a folder with a lot of files in it ( > 100) from another client.
  2. Wait for your client to find the new folder and let it start syncing

The output of your Nextcloud log in Admin > Logging:
Nextcloud Log

The output of your config.php file in /path/to/nextcloud:
config.php

The output of your Apache/nginx/system log in /var/log/____:
apache2/access.log

Hey Guys, the error logs are kinda empty. Neither syslog nor apache/error.log has anything meaningful in it. I monitored the redis-server during a sync but I couldn’t find anything wrong with it. I can post a monitor-log of redis too if wanted. Apart from that no real Errors are shown to me… Only the windows client shows a timeout Error and restarts syncing, which then works for a few seconds, and then stops again for 2-3 minutes, and starts again… the client can successfully finish the sync eventually. it just takes forever

I hope you can help me! Any help is appreciated! :ok_man:

Hello.

Restart apache2 with -vv to add verbosity.

Your problem looks like:

  • apache2 misconfiguration: check the keep alive and keep alive timeout;
  • php: check post Max size, check timeout.

Hi,

I ramped up the verbosity to Debug and my error log gets filled with lines like this while i am logged in to nextcloud via browser:
[Wed Jun 05 20:15:11.440423 2019] [authz_core:debug] [pid 25212] mod_authz_core.c(809): [client 192.168.0.2:49758] AH01626: authorization result of <RequireAny>: granted

my apache.conf values are:

Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

Maybe i should lower KeepAliveTimeout to 1… I will try tweaking these values. Any suggestions tho?

my php post_max_size should not be the problem, since i can transfer large files with no problem:

max_input_time => -1 => -1
post_max_size => 1G => 1G
upload_max_filesize => 1G => 1G
max_execution_time => 0 => 0

Thx for your help!

Some possible thing to tweak:

  • Verify consistency between php.ini
  • give more time for process
  • instead of a unlimited exe time, put limits. Sometimes the 0, -1 are the pb.

Thanks. I tried all of your ideas without success. I adjusted the values in ./fpm/php.ini ./apache2/php.ini and ./cli/php.ini.
After further inspection of the problem i can confidently say, that the downloads stop within 10 seconds in most cases, which is way before any timeout value. In some cases its even just one second :tired_face:


Just to emphasize what works

  • Download a single file > 32GB for the whole duration ~8-10minutes
  • Upload the whole directory with over 100.000s of files and a total size of over 190GB

What does not work is:

  • Download multiple files at once. For example within a directory

It seems like the server just doesnt give remaining downloads to the client. It just finishes a first batch of files (there is no fix amount of files which gets transfered - sometimes 1, sometimes 40) and then it just stops. No further files are being transfered. The files which start to download always finish completely. So no file downloads get interrupted

  • as there is no dumb testing, did you try using another browser ?

  • While downloading your files, could you start two pings:
    – the first one to your router ip
    – The second one to any responding server like google.com
    This will help to understand if your network or your router might be faulty.

  • Did you try a traceroute to your server ?

Hi guys, I did some testing to find the issue. For that i maxed the log-level to trace8 in apache. I also disabled ssl so it wouldn’t distract me in the log files.

Here is an Apache log file of a successful download of a 32GB file, which i manually aborted after 40 seconds. And another one where i tried to download a zipped folder (which i guess internally is the same as trying to downloading multiple files at once). I am also pretty sure that zipping is not the issue, cause i have the same problem with my windows client and the android nextcloud app.

Successful download - small log (more detail: large log)
16:01:38 Download Begin
16:02:18 User cancels the download

Failed download small log (more detail: large log)
16:02:38 Download Begin
16:02:39 Download stops itself
16:02:52 User cancels the download

try to disable the Gzip compression, seems your modfilter.c is complaining about some bad conf

  • another browser: Tested on Chrome 74.0.3729.169 and Firefox 67.0 (64-Bit).
    Also tested on nextcloud app for android. Same phenomenon

  • My connection from client to server is 100% stable for other applications or protocols. I can upload and download with full GBit-Speed via SFTP for example.

  • Traceroute

over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  Rock-Pi-4-A [192.168.0.150]

Trace complete.

Everything is wired via ethernet cables

maybe this is this bug !!!
https://bz.apache.org/bugzilla/show_bug.cgi?id=58901

1 Like

when i try to disable gzip it gives me

rock@Rock-Pi-4-A:~$ sudo a2dismod deflate
WARNING: The following essential module will be disabled.
This might result in unexpected behavior and should NOT be done
unless you know exactly what you are doing!
 deflate

To continue type in the phrase 'Yes, do as I say!' or retry by passing '-f': "I'm a bit unsure, but we could try maybe? °_°"

Not sure if thats the way :confused:

Your link to the bug is related to SSL in apache, which I have disabled already :frowning:


I’m starting to believe it’s a kernel issue with the Rock Pi or maybe the ethernet hardware on it, that is faulting and fooling me :sob:

you can stop deflate like this.

the kernel issue or hardware is possible, but as you are able to download large files, i dont think so.

Yeah… The issue seems to be strictly limited to apache. I can download without problem via SFTP from my server. Even when i set maximum simultaneous transfers to 20 i get no issues whatsoever.

rock@Rock-Pi-4-A:~$ sudo a2dismod deflate
[...]
Module deflate disabled.
To activate the new configuration, you need to run:
  systemctl restart apache2
rock@Rock-Pi-4-A:~$ sudo systemctl restart apache2

did not resolve the problem

sorry, without digging deep down your server, i have no more clue …

Thx for your help still! Very much appreciated :upside_down_face:

Disabling both Gzip compression AND SSL did the trick for me!
Thank you very much for your help @stratege1401! When i tested disabling Gzip (mod_deflate) yesterday, I had SSL enabled back again at that point, because i expected to have only a single problem in my nextcloud installation. That’s why i still had problems downloading. Apparently, both SSL AND deflate caused issues in my installation (so your linked bug might exist at my setup).

Now I can download with my windows client again :tada:
I still have problems, when i try to download multiple files as a ZIP file via the browser, but i dont really care about that :man_shrugging: I’m just happy, that the windows client works as it should :star_struck:
… …
… (for now :face_with_monocle:)

:sweat_smile:

No problem.

Concerning the bug, i hope next apache2 release will have a fix !

1 Like

Hi, I just wanted to reiterate, that I again can’t download multiple files via windows client. I don’t know what’s happening. The download just stops within seconds, when syncing. I guess i will reinstall on a fresh card again and see what happens.

I tried multiple things, as using lighttpd or nginx. I messed around with all the caching settings, ssl and any options i could think of. It just didnt work. I think the dedicated Ubuntu Server image for Rock Pi 4 is just messed up.

I now have the same setup on a newly purchased Intel Nuc running stock Ubuntu Server 18.04.02 LTS and it works flawlessly. So I can’t recommend the Rock Pi for Nextcloud unfortunately.