Any way to force chunking for uploads made from the web client?

Hi there,

I’ve had Nextcloud installed for a while now on my Unraid server through docker, i mostly used it for the apps (calender, deck and so on) and some occasional small files transfers until now and it’s been working fine for that.

Recently though, i’ve started using it for larger and more frequent transfers and i’ve run into an issue that prevented me from uploading large files (over 100mb), both throught the web UI and the app.
After a bit of research it seems that the issue has to do with nextcloud not chunking files and hitting cloudflare’s limit of 100mb transfer per connection.

I was able to resolve the issue in the app with this solution and now I’m able to upload large files without issues using synched folder on my computer. However, the issue persists on the web client, where i find myself having to manually compress and divide into < 100mb parts the large files i am trying to upload, which is not ideal.

So I am wondering wether there is a way to force the chunking of uploads to a size under 100mb for the whole server, including the web client, in the same way i was able to do for the app ? I haven’t been able to find one yet.

Thanks for the help !

I didn’t test:

https://docs.nextcloud.com/server/stable/admin_manual/configuration_files/big_file_upload_configuration.html#adjust-chunk-size-on-nextcloud-side

would be great if you confirm this works.

So i tried running the following command to adjust chunk size in nextcloud’s container console:

sudo -u www-data php occ config:app:set files max_chunk_size --value 20971520

but i get this error:

sudo: unknown user www-data
sudo: error initializing audit plugin sudoers_audit

Check to see what the user is on your distro. My Alma system (which is the same for all RHEL derivatives and I expect Fedora) has the user apache for running httpd rather than www-data.

1 Like

How can i identitfy what user is on my distro ?

I tried running the command with every user listed on this page (so apache, www-data, http and wwwrun) but i get the same error as before every time.

I vaguely remember reading on a post earlier that it could be necessary on some nextcloud installs to install sudo, though i can’t for the life of me find that page again or the procedure to do so…
Could this be the issue ? If so how do i install sudo ?

Let’s nail that “unknown user” message first before worrying about NextCloud (at which I’m a novice). The “quick and dirty” way is to issue the following two commands:

$ sudo grep -i http /etc/passwd
$ sudo grep -i apache /etc/passwd

It might also help if you could check /etc/os-release or if that is not found /etc/*release and report back the distro name. Look for lines starting “ID=” and “ID_LIKE=”.

I’m not familiar with your setup, and if you have a non-standard ID&A then it’s possible that the above might fail. Likewise docker shouldn’t be an issue, but I wouldn’t rule it out yet.

1 Like

The grep commands result in no output at all in terminal :

I am using linuxserver’s docker image, the docker is up to date and my Nextcloud install is on version 24.0.8. My system is ruinning Unraid 6.11.1

I couldn’t find anything containig release in /etc/ however, i retried using the command for setting chunk size but removed the first bit “sudo -u www-data php” :

occ config:app:set files max_chunk_size --value 20971520

And this worked, i got a confirmation that chunk size was modified… Not really sure what to make of this.

Nevertheless, i tested transferring a few big files but at first transfers would fail generally around 1.2GB. I tested different chunk sizes and weirdly enough, by changing the chunk size to 200MB (which should have been beyond cloudflare’s limit) it worked and i was able to upload a 2.6GB file through the web client.

I’ll be doing more testing to see how stable that is or if that was just a one off, but it seems that the issue might not have been what i thought originally.

Ah, if you were inside the docker image then perhaps it hid the ID&A and release information. Sorry if this was a red herring, but at least you’ve managed to get it sorted. I’ve been looking at Unraid for you, and it does seem to be a way of hiding things from the user. Unraid is based on Slackware and for that the Apache user is apache as far as I can tell from an internet search. I don’t currently have a slackware VM running on my machine. For reference then, your “sudo” command should have started:

$ sudo -u apache

The main thing is though that you’ve solved your problem, best wishes, and this is the point I bow out.

1 Like