Upload only really starts after the progress bar is full

ℹ️ Support
1

Nextcloud version : 29.0.0
Operating system and version : Debian 11.9 (x86_64)
Apache or nginx version : Apache/2.4.59 / nginx as Proxy/1.25.5
PHP version : 8.2

The issue you are facing:

When trying to upload a file the progress bar takes the expected time until finished. Since large files take longer filling the bar. When it’s filled it’s stuck at “a few seconds left” and then starts uploading the file which can take a while for large files. Now I had to already set up the timeout otherwise the upload would even stop after 5 minutes and give an error.
Is this the first time you’ve seen this error? : N

Steps to replicate it:

  1. Upload a large file
  2. Wait until progress bar is filled
  3. Go on ftp and see the file is just starting to get uploaded after the bar is filled

The output of your Nextcloud log in Admin > Logging:

|Warnung|no app in context|Exception Transaction took 2.4642639160156s|"20.05.2024, 10:25:56"||
| --- | --- | --- | --- | --- |
|Warnung|no app in context|Exception Transaction took 2.5775899887085s|"20.05.2024, 10:25:00"||
|Warnung|no app in context|Exception Transaction took 3.6030189990997s|"20.05.2024, 10:23:58"||
|Warnung|no app in context|Exception Transaction took 3.1955461502075s|"20.05.2024, 10:22:57"||
|Warnung|no app in context|Exception Transaction took 1.2162249088287s|"20.05.2024, 10:22:06"||
|Warnung|no app in context|Exception Transaction took 1.9353880882263s|"20.05.2024, 10:18:57"||
|Warnung|no app in context|Exception Transaction took 4.4737200737s|"20.05.2024, 10:18:10"||
|Warnung|no app in context|Exception Transaction took 2.3284709453583s|"20.05.2024, 10:17:56"||
|Warnung|no app in context|Exception Transaction took 2.3585329055786s|"20.05.2024, 10:17:10"|

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'trusted_domains' => 
  array (
    0 => 'localhost',
  ),
  'datadirectory' => '/home/nextcloud/web/public_html/data',
  'dbtype' => 'mysql',
  'version' => '29.0.0.19',
  'overwrite.cli.url' => 'http://localhost',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'installed' => true,
  'instanceid' => 'oc7bz6x64kvf',
  'mail_from_address' => 'info',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_smtpport' => '465',
  'mail_smtpauth' => 1,
  'mail_smtpsecure' => 'ssl',
  'enable_previews' => true,
  'enabledPreviewProviders' => 
  array (
    0 => 'OC\\Preview\\HEIC',
    1 => 'OC\\Preview\\TIFF',
    2 => 'OC\\Preview\\Movie',
    3 => 'OC\\Preview\\JPEG',
    4 => 'OC\\Preview\\PNG',
    5 => 'OC\\Preview\\Image',
  ),
  'app_install_overwrite' => 
  array (
    0 => 'groupfolders',
    1 => 'vibeetoee',
    2 => 'extract',
  ),
  'loglevel' => 2,
);
2

Hi @Phil_C

“Where?” you go on ftp?

Not everybody connects with ftp to his server. :wink:

You seem to have somehow not fully understood the way nextcloud handles and processes (large) updates and are drawing incorrect conclusions from it.

Nextcloud doesn’t just take a file and then write it on the server in its place, but rather it is chunked and stored in temporary directories and then reassembled on the server when everything is there. You obviously interpret this as the actual start of the upload, but by then the upload itself is usually already completed.

Please read these resources to get a better understanding how it works and how you can tweek around:

Much and good luck,
ernolf

1 Like
3

yes I have read that. I still doubt that it takes 1 hour to move already uploaded files into place ? I see the chunk folder but its also only created after the progress bar is run through
Adding to that I set the max chunk size to 0 and now there are no chunks at all so it is indeed exactly storing it where I upload and still only starts uploading after the progress bar is completely run through.

4

No, that conclusion may again been drawn too quickly.

What are the values of these PHP directives (if set) in either /etc/php/8.2/apache2/php.ini or /etc/php/8.2/fpm/php.ini, (depending on which SAPI your apache is configured to speak with php):

max_execution_time
max_input_time
memory_limit
post_max_size
upload_tmp_dir
upload_max_filesize
default_socket_timeout

ernolf

1 Like
5

max_execution_time = 7200
max_input_time = 7200
memory_limit = 1024M
post_max_size = 50000M
upload_tmp_dir = /var/bigfiles/
upload_max_filesize = 50000M
default_socket_timeout = 60

6

OK, some settings are OK. My recommendation would be not to define so many restraints though. You can always lower the restrictions if your server get oom.

max_input_time = -1
memory_limit = -1
post_max_size = 0
upload_max_filesize = 64G
default_socket_timeout = -1

I have absolutely no idea if this will fix your problem, but it’s never a bad idea to give the server more room to breathe.:wink:

Much and good luck,
ernolf

7

Thanks I mean the uploads aren’t failing. It’s just that it only starts uploading after the progress bar is through which already takes like idk 1 hour depending on the size of the file. And then it starts the real upload? So im curios what is happening in the meantime and why the progress bar giving false information when its reached the end saying “few seconds left” but the whole upload is now being in the progress.

8

Ok I found the issue. It was clamav blocking my whole system.

9

Well then at least you know that the virus scanner is working. :white_check_mark:

Clam checks and blocks until it has finished checking. If you don’t want that, you have to configure Clam accordingly.
I have an extra dedicated AV host that takes over the scan so that it doesn’t happen on the same machine. That makes a huge difference.

I am happy that you found the culprit!

Much and good luck,
ernolf