How to debug a failing upload

🍱 Features & apps flowupload
1

Hi everyone,
for some reason an upload of a file of about 5GB keeps failing, and I want to debug the reason.
Do you have any suggestion on how to do that?

My system is a raspberry pi 4, raspbian buster and the last stable version of nextcloud.

Thanks

2

Hello.

Have you changed something recently?
Have you been able to upload such large files before?

I had a problem a few months ago when I had put /tmp in ram using tmpfs. Suddenly large files stopped working.
I then changed the directory where temporary files are stored during upload.
However, now when i try to see where temporary files are stored they get stored in

  • chosen directory (if uploaded using ios app)
  • /<nextcloud-data>/<user>/uploads (if uploaded using web gui).
    When I had the problems I was using Ubuntu 18+NC 18, today when I did the tests I used Ubuntu20+NC 20, so temporary location may have changed. It may also be a php setting.

Unfortunately I do not know any way to debug this issue.

3

Hi,

what comes to my mind is only an update to the latest version of nextcloud and maybe of the plugin itself from the previous.

Yes, before I was able to upload files… what’s wired is no evidence of any error in the log files.
What I saw is just the upload of the various file chunks, and that’s it. Chunks didn’t get reassembled, with no message of error or warning. That’s why I wanted to understand how to enable a kind of debug for the plugin.

Is there any configuration I can tweak? As far as I can see, there’s nothing to set.

Thanks

4

I just did a test by uploading (syncing) a file using the desktop sync app.
In my test (perhaps setting dependent):

  • A temporary file is saved to /tmp and is called phpXXXXXX where X is some random character.
  • The chunk (100 Mb) is then moved into /<nextcloud-data>/<user>/uploads/
    You can see these kind of logs in the apache log:
192.168.1.50 - - [23/Feb/2021:12:10:23 +0900] "PUT /remote.php/dav/uploads/user/4017789609/00000025 HTTP/1.1" 201 845 "-" "Mozilla/5.0 (Linux) mirall/2.6.5stable (build 20200710) (Nextcloud)"
  • After uploading it will be concatenated at the correct location.

If it is a large file (Gbytes) it may take some time for it to concatenate the chunks.

Does it matter if you upload using web, the sync app or mobile app?
Have you found the file size limit when an upload always fails or succeeds, or is it of random nature? (have you tried a 4.8 Gb file for example)

5

Hello Marco!

Today I also had a failing upload from the app on my ipad. It was working before, but since that I have modified my system a lot (upgrades etc)…

The app version is 3.2.0.30 and my NC server is 20.0.8, Ubuntu 20.04 and php 7.4.

Anyway, the file was about 500 MB.
I noticed that the file was saved to both /tmp (default php upload directory) as /tmp/phpxw9sqS and in my nextcloud directory as IMG_0739.MOV.ocTransferId1515077066.part. My /tmp partition is only 256 MB so when it got full the temporary files were removed and the upload retried.
The logs in nextcloud admin settings gave these errors:

Error	no app in context	Sabre\DAV\Exception\BadRequest: Expected filesize of 551250667 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 268414976 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.
Error	PHP	Error: fread(): write of 8192 bytes failed with errno=28 No space left on device at /var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php#91

The solution for me was to change the temporary upload directory for php in php.ini (/etc/php/7.4/fpm/php.ini in my case).

Have you checked your php settings?
Which nc version are you running?

6

Hi, actually my php.ini is already set with pretty large limits, and the default /tmp has about 10GB free…however I didn’t have a custom upload folder.
I’ll set it now and do some tests.

Thanks for now, I’ll come back very soon!

Cheers

7

Ah sorry, I forgot to mention I’m running the latest stable version (20.0.8) and the flow upload plugin is 1.1.2.

Thanks

8

So is it only failing when using the Flow upload plugin?
I assumed it was failing for all kinds of uploads.

9

Hi, it works now :slight_smile:
Cheers