Rclone do not upload large file and error only at timeout

ℹ️ Support
1

Nextcloud version (eg, 20.0.5): 24.0.5.1
Operating system and version (eg, Ubuntu 20.04): Bedian 11 on a Raspberry pi
Apache version (eg, Apache 2.4.25): 2.4.54
PHP version (eg, 7.4): 7.4

The issue you are facing:

I have a rclone job on another PC that copies its files to nextcloud from time to time.
But yesterday I’ve noticed that a 1.4G file was not sent. I tried different things and discovered that only big files seems to be impacted.
Then I proceed to change all variables that might impact the file size limit (mostly based on this doc : Uploading big files > 512MB — Nextcloud latest Administration Manual latest documentation).
But nothing has worked. And one thing is strange I can fully upload bigger files on the web GUI.

The error is :
`

rclone copy file.mp4 "remote:.path/" --no-check-certificate --progress --log-level=DEBUG


2022/09/18 13:48:48 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "copy" "file.mp4" "remote:.path/" "--no-check-certificate" "--progress" "--log-level=DEBUG"]
2022/09/18 13:48:48 DEBUG : Creating backend with remote "file.mp4"
2022/09/18 13:48:48 DEBUG : Using config file from "/home/automation/.config/rclone/rclone.conf"
2022/09/18 13:48:48 DEBUG : fs cache: adding new entry for parent of "file.mp4", "/home/automation/video_manager"
2022/09/18 13:48:48 DEBUG : Creating backend with remote "remote:.path/"
2022/09/18 13:48:48 DEBUG : fs cache: renaming cache item "remote:.path/" to be canonical "remote:.path/"
2022-09-18 13:48:49 DEBUG : file.mp4: Need to transfer - File not found at Destination
2022-09-18 13:50:34 ERROR : file.mp4: Failed to copy: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>413 Request Entity Too Large</title>
</head><body>
<h1>Request Entity Too Large</h1>
The requested resource does not allow request data with PUT requests, or the amount of data provided in
the request exceeds the capacity limit.
<hr>
<address>Apache/2.4.54 (Debian) Server at remote Port 443</address>
</body></html>
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
  <s:message>Taille du fichier attendue : 1433769082 octets mais taille du fichier lue (depuis le client Nextcloud) et écrit (dans le stockage Nextcloud) : 0 octet. Cela peut être un problème de réseau au niveau du client ou un problème de stockage au niveau du serveur.</s:message>
</d:error>: 413 Request Entity Too Large
2022-09-18 13:50:34 ERROR : Attempt 1/3 failed with 1 errors and: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>413 Request Entity Too Large</title>
</head><body>


2022-09-18 13:50:35 DEBUG : file.mp4: Need to transfer - File not found at Destination
Transferred:          284k / 1.336 GBytes, 0%, 2.429 kBytes/s, ETA 6d16h7m52s
Transferred:            0 / 1, 0%
Elapsed time:      1m59.3s
Transferring:
 * file.mp4:  0% /1.335G, 0/s, -

`

2

Same here

3

Hello, Bedian 11 on a Raspberry pi ( are u sleeping well ? )

just tested rclone… tested ryzen5 - debian11 - NC24 stable updated. seem to work for me.
Your error seem to be a locked files problem.

are u a 32 bits host. Could be a system files limitation.

4

After various tests, it turned out that my uploads were limited to exactly 1GB. It was the LimitRequestBody parameter, the limiting factor.

So I added at the end of my “nextcloud-le-ssl.conf” file

LimitRequestBody 0

In my case, this must be added in “nextcloud-le-ssl.conf” and not in “nextcloud.conf”.

1 Like
5

Thank for your test :wink:

Citation
Hello, Bedian 11 on a Raspberry pi ( are u sleeping well ? )

haha indeed I did not see that misspelling.

Citation
Your error seem to be a locked files problem.

I don’t think it’s a locked file problem, because if I change the name of the file the problem remains the same :confused:

Citation
are u a 32 bits host. Could be a system files limitation.

Maybe but it’s strange I upgraded my “old” 32bit debian 10 to the 64bit debian 11

6

Thanks for your time :smiley:

I tested :

<Directory /pathHidden>
                AllowOverride All
                LimitRequestBody 0
</Directory>
<IfModule mod_headers.c>
                Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
                LimitRequestBody 0
 </IfModule>

But no look the problem is still there

7

Finally, it seems to work.

Since I use a raspberry pi zero for the rclone client and relatively big files (about 1,5G) the first two minutes, the raspberry pi zero has to compute some verification.
That lured me into thinking that modification does not work. :smiley:

8

According to my tests, I had to put it at the very end of the file for it to work. Maybe it’s not very clean, but it works… :wink: Perhaps an apache specialist will come by…

<IfModule mod_ssl.c>
<VirtualHost *:443>
     ServerAdmin xxxxxxxxxxxxxxx
     DocumentRoot xxxxxxxxxxxxxxxxxx
     ServerName xxxxxxxxxxx

     Alias /nextcloud "xxxxxxxxxxxxxxxxxxx"

     <Directory xxxxxxxxxxxxxxxxxxxxxx>
        Options +FollowSymlinks
        AllowOverride All
        Require all granted
          <IfModule mod_dav.c>
            Dav off
          </IfModule>
          <IfModule mod_headers.c>
        Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
          </IfModule>
        SetEnv HOME xxxxxxxxxxxxxxxxx
        SetEnv HTTP_HOME xxxxxxxxxxxxxxxxx
     </Directory>

     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined

LimitRequestBody 0
SSLCertificateFile xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
SSLCertificateKeyFile xxxxxxxxxxxxxxxxxxxxxxxxxx
Include xxxxxxxxxxxxxxxxxxxxx
</VirtualHost>
</IfModule>