Nextcloud 16.0.4 "Not enough free space"

Nextcloud version (eg, 12.0.2): 16.0.4
Operating system and version (eg, Ubuntu 17.04):Synology DSM 6.2
Apache or nginx version (eg, Apache 2.4.25): Nginx 1.15.7
PHP version (eg, 7.1):7.2

The issue you are facing:
From the web UI, an upload of any file size receives the popup “Not enough free space”. The volume I’m uploading to has nearly 6TB free and I’ve verified file permissions multiple times.

The nginx logs are showing something interesting. The ajax call to getstoragestats.php is returning a 302 redirection to /apps/files/

root@localhost:/var/log/nginx# tail -f access.log | grep storage
127.0.0.1 - - [13/Sep/2019:11:35:14 -0400] "GET /index.php/apps/files/ajax/getstoragestats.php?dir=%2F HTTP/2.0" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:35:14 -0400] "GET /index.php/apps/files_external/userglobalstorages/2?testOnly=false HTTP/2.0" 200 199 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:35:14 -0400] "GET /index.php/apps/files_external/userglobalstorages/1?testOnly=false HTTP/2.0" 200 197 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:35:14 -0400] "GET /index.php/apps/files_external/userglobalstorages/4?testOnly=false HTTP/2.0" 200 201 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:35:15 -0400] "GET /index.php/apps/files_external/userglobalstorages/3?testOnly=false HTTP/2.0" 200 197 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:35:15 -0400] "GET /index.php/apps/files_external/userglobalstorages/6?testOnly=false HTTP/2.0" 200 197 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:
1.) Attempt to upload a file of any size.

The output of your Nextcloud log in Admin > Logging: No logs are produced from this event (debug or otherwise).

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => '',
  '',
  'secret' => '',
  'trusted_domains' =>
  array (
    0 => '',
  ),
  'datadirectory' => '/volume1/Nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '16.0.4.1',
  'overwrite.cli.url' => '',
  'dbname' => '',
  'dbhost' => '127.0.0.1:3307',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => '',
  'dbpassword' => '',
  'installed' => true,
  'mail_domain' => '',
  'mail_from_address' => '',
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'tls',
  'mail_smtphost' => '',
  'mail_smtpauth' => 1,
  'mail_smtpport' => '587',
  'mail_sendmailmode' => 'smtp',
  'mail_smtpname' => '',
  'mail_smtppassword' => '',
  'twofactor_enforced' => 'false',
  'twofactor_enforced_groups' =>
  array (
    0 => 'admin',
  ),
  'twofactor_enforced_excluded_groups' =>
  array (
  ),
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
);

The output of your Apache/nginx/system log in /var/log/____:

127.0.0.1 - - [13/Sep/2019:11:52:42 -0400] "GET /apps/files/?dir=/&fileid=7 HTTP/2.0" 303 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:42 -0400] "GET /apps/files/?dir=/ HTTP/2.0" 200 7873 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:43 -0400] "GET /js/files/merged-index.js?v=fe23deb5-3 HTTP/2.0" 200 90066 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:43 -0400] "GET /apps/theming/styles?v=3 HTTP/2.0" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:43 -0400] "GET /js/activity/activity-sidebar.js?v=fe23deb5-3 HTTP/2.0" 200 5038 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:43 -0400] "GET /js/gallery/scripts-for-file-app.js?v=fe23deb5-3 HTTP/2.0" 200 56009 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:43 -0400] "GET /apps/accessibility/js/accessibility?v=0 HTTP/2.0" 200 71 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:43 -0400] "GET /apps/theming/js/theming?v=3 HTTP/2.0" 200 215 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:43 -0400] "GET /avatar/user/32?v=1 HTTP/2.0" 200 1722 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:44 -0400] "GET /svg/core/actions/more?color=000&v=1 HTTP/2.0" 499 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:44 -0400] "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/2.0" 200 392 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:44 -0400] "GET /svg/core/places/contacts?color=000&v=1 HTTP/2.0" 200 463 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:44 -0400] "GET /svg/core/logo/logo?color=fff&v=1 HTTP/2.0" 200 190 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:44 -0400] "GET /svg/files/folder?color=fff&v=1 HTTP/2.0" 200 252 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:44 -0400] "GET /svg/core/actions/star-dark?color=fff&v=2 HTTP/2.0" 200 207 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:45 -0400] "GET /svg/files/recent?color=fff&v=1 HTTP/2.0" 200 213 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:45 -0400] "GET /svg/core/actions/triangle-s?color=fff&v=1 HTTP/2.0" 200 145 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:45 -0400] "GET /svg/files/external?color=fff&v=1 HTTP/2.0" 200 225 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:45 -0400] "GET /svg/files/share?color=fff&v=1 HTTP/2.0" 200 211 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:45 -0400] "GET /svg/files/delete?color=fff&v=1 HTTP/2.0" 200 220 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:45 -0400] "GET /svg/core/actions/settings-dark?color=fff&v=1 HTTP/2.0" 200 450 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:46 -0400] "GET /svg/core/actions/toggle-pictures?color=fff&v=1 HTTP/2.0" 200 166 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:46 -0400] "GET /cron.php HTTP/2.0" 200 46 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:46 -0400] "GET /apps/gallery/config?extramediatypes=1 HTTP/2.0" 200 102 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:46 -0400] "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/2.0" 200 392 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:46 -0400] "PROPFIND /remote.php/dav/files/user/ HTTP/2.0" 207 5580 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:46 -0400] "GET /svg/core/actions/quota?color=fff&v=1 HTTP/2.0" 200 215 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /apps/recommendations/api/recommendations HTTP/2.0" 200 292 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /svg/core/places/home?color=fff&v=1 HTTP/2.0" 200 168 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /svg/core/actions/search?color=fffffe&v=1 HTTP/2.0" 200 243 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /svg/core/actions/add?color=fff&v=1 HTTP/2.0" 200 176 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /index.php/apps/files/ajax/getstoragestats.php?dir=%2F HTTP/2.0" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /apps/theming/img/core/filetypes/folder-external.svg?v=3 HTTP/2.0" 200 344 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /svg/core/actions/triangle-n?color=fff&v=1 HTTP/2.0" 200 147 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:47 -0400] "GET /svg/core/actions/more?color=fff&v=1 HTTP/2.0" 200 227 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:48 -0400] "GET /apps/theming/img/core/filetypes/file.svg?v=3 HTTP/2.0" 200 228 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:48 -0400] "GET /ocs/v1.php/apps/files_external/api/v1/mounts?format=json HTTP/2.0" 200 226 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:48 -0400] "GET /index.php/apps/files/ HTTP/2.0" 200 7882 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:48 -0400] "GET /index.php/apps/files_external/userglobalstorages/2?testOnly=false HTTP/2.0" 200 199 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:48 -0400] "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/2.0" 200 74 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:48 -0400] "GET /index.php/apps/files_external/userglobalstorages/3?testOnly=false HTTP/2.0" 200 197 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:48 -0400] "GET /index.php/apps/files_external/userglobalstorages/4?testOnly=false HTTP/2.0" 200 201 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:49 -0400] "GET /index.php/apps/files_external/userglobalstorages/1?testOnly=false HTTP/2.0" 200 197 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
127.0.0.1 - - [13/Sep/2019:11:52:49 -0400] "GET /index.php/apps/files_external/userglobalstorages/6?testOnly=false HTTP/2.0" 200 197 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"

Have you checked the user settings? As an admin you can attribute a quota for each user.

Yes. My user is set to unlimited, but other users with a specific limit (100G and 10G) see the same popup.

Edit: also, uploads are working find in the mobile apps.

My feeling is that this is an nginx config mishap, nextcloud config mishap, or code bug because of the 302 error returned when the browser tries retrieve the storage size.

Synology uses a reverse proxy to host sites and makes their configurations a maze of includes, but as far as I can tell the nginx config doesn’t perform a 302 redirect anywhere.

I’ve been through the nextcloud docs on nginx and instance configs a few times. The user config I used for my nginx instance was from an early version of 15 so I updated to the current recommended config from 16 but saw no change. I also can’t find anything in the nextcloud config that could affect this.

Can you check with df -h if the device was probably mounted and the available free space? You perhaps also spot a partition which is full. Keep in mind that some things during upload can be put in the tmp folder.

My nextcloud instance and data directory are on volume1

df -h
Filesystem         Size  Used Avail Use% Mounted on
/dev/md0           2.3G  1.3G  918M  59% /
none               3.9G     0  3.9G   0% /dev
/tmp               3.9G  980K  3.9G   1% /tmp
/run               3.9G   15M  3.8G   1% /run
/dev/shm           3.9G  1.6M  3.9G   1% /dev/shm
none               4.0K     0  4.0K   0% /sys/fs/cgroup
cgmfs              100K     0  100K   0% /run/cgmanager/fs
/dev/md4           216G   23G  194G  11% /volume2
/dev/vg1000/lv      15T  8.7T  5.7T  61% /volume1
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/d1c05f4021b2c695deae4e7d12d84bf63dad0ee998a96f53d5c6a625b95775d7
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/35cb878ed1b79517bea9d4e0a396c5abf03f18a4580a6bcc119a3cdf6c8c247b
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/8ea1a89b00859cad2be49c2ce230e6765ed69237ed05fa2786d12afa64153062
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/50849298a63a5126a668e94c724b0010a186cd72caf123eb22361f20b5e9c3fd
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/0503848520fc93e086d1f7ab757b139f16619bb7c54a83b8e7be089d8a907014
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/07686df43c5bdeac1d6bb39f5b5f8c18a17d73c4df17a707537a91cbd1114e88
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/088f57f068e0f9c6dd39ea2f1cc59422584a8347c5990538663b85d0ca93affa
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/02dad23eaf4f0f03b67405fedafa8e62591b95e19175d3a281df4b192438f23e
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/d64393ef69fceaa7af24fb5c41bee18328abd64072cc5fa931bad69e1533e8ce
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/0c17eecf87e11146aefd49460525f88921a44b377701bde02b158445f6f4805f
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/8d808041d0476793498afc0b9f71ccf5610321c1c0657eb3d06a90af8b02aab9
/volume1/@Backup@   15T  8.7T  5.7T  61% /volume1/Backup
none               216G   23G  194G  11% /volume2/@docker/aufs/mnt/c38531b239dd9d69b94830bc7855ddf521c6882bb261e46aa42cfbaab2dc5646

Is there anyway to debug where the 302 redirect is coming from for the ajax call to getstoragestats.php?

I know the file doesn’t exist. It seems to be a route on appinfo but I can’t find the code that serves it. Are the routes stored in the database somehow? Is it possible something wasn’t entirely correct from the upgrade between 15 and 16?

The problem is the way synology sets up virtual hosts.

As part of the vhost config, synology does the following:

    location ~* \.(php[345]?|phtml)$ {
        fastcgi_pass unix:/run/php-fpm/php-xxxxxxxxxxxxxxxxxxxxxxxxxxx.sock;
        fastcgi_param HOST "host";
        include fastcgi.conf;
    }

    include /usr/local/etc/nginx/conf.d/xxxxxxxxxxxxxxxxxxxxxxxxxxx/user.conf*;

The only config I have control over is the user.conf. Since the nextcloud config is looking for php files using a regex match, the nextcloud rule is ignored and the synology rule is used.

Not an elegant fix, but you can use the nginx rules against itself. If you wrap all of the prefix and pattern matching rules of the nextcloud nginx config in a location / block, you can bypass the server level regex match.

location / {
//nextcloud rules, including the location / rewrite
}