Local storage Question

support
, ,
#10

Stephan,

For the ls -l /var/ command I get:

ncadmin@cloud:~$ ls -l /var/

total 64

drwxr-xr-x  2 root root   4096 Feb  9 06:25 backups

drwxr-xr-x 16 root root   4096 Nov 24 19:30 cache

drwxrwxrwt  2 root root   4096 Feb 12 07:40 crash

drwxr-xr-x 54 root root   4096 Nov 24 19:44 lib

drwxrwsr-x  2 root staff  4096 Apr 24  2018 local

lrwxrwxrwx  1 root root      9 Oct 16 11:55 lock -> /run/lock

drwxrwxr-x 14 root syslog 4096 Feb 12 07:40 log

drwxrwsr-x  2 root mail   4096 Nov 24 19:44 mail

drwxr-xr-x  4 root root   4096 Feb  8 22:19 NCBACKUP

drwxr-xr-x  5 root root   4096 Feb  8 22:19 NCBACKUP-OLD

drwxr-xr-x  2 root root   4096 Jul 24  2018 opt

lrwxrwxrwx  1 root root      4 Oct 16 11:55 run -> /run

drwxr-xr-x  3 root root   4096 Feb  8 22:31 scripts

drwxr-xr-x  2 root root   4096 Jul 19  2018 snap

drwxr-xr-x  6 root root   4096 Nov 24 19:44 spool

drwxrwxrwt  6 root root   4096 Feb 12 08:39 tmp

drwx------  3 root bin    4096 Feb  8 21:29 webmin

drwxr-xr-x  4 root root   4096 Feb  8 22:20 www
#11

How about to run this command instead:

sudo grep -i datadirectory /var/www/nextcloud/config/config.php

this will give you access to config file if it is in /var/www/nextcloud. Output will show you where is your data saved. Example:

 sudo grep -i datadirectory /var/www/nextcloud/config/config.php
'datadirectory' => '/var/www/nextcloud/data',

Did you install NC with snap, docker or manually?
I suppose you use snap, in this case if you run ps -ef | grep nextcloud command you will see similar output as in this comment: How to: change data directory in nextcloud snap. In this case your data folder is probably under: /var/snap/nextcloud/common/nextcloud/data.

#12

Stephan, I ran the command and get:
ncadmin@cloud:~$ sudo grep -i datadirectory /var/www/nextcloud/config/config.php
[sudo] password for ncadmin:
‘datadirectory’ => ‘/mnt/ncdata’,

the df command shows:

ncadmin@cloud:~$ df
Filesystem                     1K-blocks    Used Available Use% Mounted on
udev                             1977132       0   1977132   0% /dev
tmpfs                             401584    1284    400300   1% /run
/dev/mapper/nextcloud--vg-root  39990560 9435272  28494184  25% /
tmpfs                            2007904       8   2007896   1% /dev/shm
tmpfs                               5120       0      5120   0% /run/lock
tmpfs                            2007904       0   2007904   0% /sys/fs/cgroup
ncdata                           3378304 3378304         0 100% /mnt/ncdata
tmpfs                             401580       0    401580   0% /run/user/1000

This installation was a VM purchase from @enoch85. I though this would be the easiest way to get running for a Windows guy like me.

#13

Would it hurt to follow this link? I don’t think it will help with NC but good general practice?

Thanks!

#14

I have 6.2gb of data I want to keep in sync with NC After 2.5GB uploaded I get “Your storage is full files can not be updated or synced anymore!”

It might be that the appdata folder is very large, or that you (or your users) didn’t remove files stored in trashbin.

Set trashbin retention in config.php. You can find examples here: https://docs.nextcloud.com/server/15/admin_manual/configuration_server/config_sample_php_parameters.html#deleted-items-trash-bin

That’s a start at least.

#15

Hi Daniel,

Thanks for chiming in again!.. I will review the doc but have already ran:

sudo -u www-data /var/www/nextcloud/occ trashbin:cleanup --all-users

sudo -u www-data /var/www/nextcloud/occ versions:cleanup

and cleaned this up.

Am I missing something?

Thanks

#16

Sorry, I don’t know all the ‘occ’ commands from the top of my head, but you can always check the docs.

Also, when using the VM you can use ‘nextcloud_occ actual:command’ instead if writing the whole path. :slight_smile:

#17

So I was able to peck around the file system till i found where the data is kept. I thought there would be extraneous file not synced but its all good. I also searched around looking for appdata solutions. I came across this, Screwed up data/appdata_instanceid

Think following this will fix my issue? Right now my system is completely down :frowning:

Thanks!

#18

Is your problem in appdata directory? Run command to find out what is your biggest folder is (ask ubuntu):

du -ah --max-depth=1  /mnt/ncdata | sort -hr

than you can check what is the biggest folder under the biggest folder and decide what to do next.

#19

@gas85, Thanks for your help! I was about to give up. The output from the command is:

ncadmin@cloud:~$ sudo du -ah --max-depth=1  /mnt/ncdata | sort -hr
[sudo] password for ncadmin:
3.3G    /mnt/ncdata
2.6G    /mnt/ncdata/dtorres
633M    /mnt/ncdata/appdata_ocm2710gjvgh
11M     /mnt/ncdata/ncadmin
11M     /mnt/ncdata/jtorres
11M     /mnt/ncdata/datorres
213K    /mnt/ncdata/files_external
65K     /mnt/ncdata/nextcloud.log
4.5K    /mnt/ncdata/updater.log
4.5K    /mnt/ncdata/.htaccess
512     /mnt/ncdata/updater-ocm2710gjvgh
512     /mnt/ncdata/.ocdata
512     /mnt/ncdata/index.html

From what i know, I don’t see any problems with space. I am also now getting this error after logging in:

Internal Server Error
The server was unable to complete your request.
If this happens again, please send the technical details below to the server administrator.
More details can be found in the server log.
Technical details
Remote Address: XX.XX.XX.XX
Request ID: fWbbCMp5M41PsU7fz9k1

Ugh!! So frustrating!

#20

Update

I’m getting cert errors, due to network connection problems over port 80. Should be getting fixed very soon.

File sync is oddly working fine :slight_smile:

#21

Thats funny, because your partition has no space left (100% being used):

I believe that this user (dtorres) has used most of your space.

Also appdata looks a little bit big. I bet you have previews enabled and if you check inside of appdata folder preview will be the biggest one. Example of command under appdata folder:

du -ah --max-depth=1 . | sort -hr
27G	./preview
27G	.
2.9M	./js
1.4M	./appstore
704K	./css
664K	./avatar
228K	./theming
204K	./dav-photocache
124K	./identityproof
20K	./external
8.0K	./richdocuments

As you can see my previews folder is huge, but this does not disturbs me --> Previews / data size

#22

OK here is the before:

ncadmin@cloud:~$ sudo du -ah --max-depth=1  /mnt/ncdata | sort -hr
[sudo] password for ncadmin:
2.8G    /mnt/ncdata
2.7G    /mnt/ncdata/dtorres
129M    /mnt/ncdata/appdata_ocm2710gjvgh
11M     /mnt/ncdata/ncadmin
11M     /mnt/ncdata/jtorres
11M     /mnt/ncdata/datorres
213K    /mnt/ncdata/files_external
65K     /mnt/ncdata/nextcloud.log
4.5K    /mnt/ncdata/updater.log
4.5K    /mnt/ncdata/.htaccess
512     /mnt/ncdata/updater-ocm2710gjvgh
512     /mnt/ncdata/.ocdata
512     /mnt/ncdata/index.html

I am very new to NextCloud and Linux in general. I edited the config.php file, added:

  'enable_previews' => true,
    'enabledPreviewProviders' => 
    array (
      0 => 'OC\\Preview\\PNG',
      1 => 'OC\\Preview\\JPEG',
      2 => 'OC\\Preview\\GIF',
      3 => 'OC\\Preview\\BMP',
      4 => 'OC\\Preview\\XBitmap',
      5 => 'OC\\Preview\\Movie',
      6 => 'OC\\Preview\\PDF',
      7 => 'OC\\Preview\\MP3',
      8 => 'OC\\Preview\\TXT',
      9 => 'OC\\Preview\\MarkDown',
    ),
    'preview_max_x' => 1080,
    'preview_max_y' => 1920,
),
);

Then ran :
sudo -u www-data /var/www/nextcloud/occ config:app:set --value=“32 64 1024” previewgenerator squareSizes

Responce was:
Config value squareSizes for app previewgenerator set to 32 64 1024

sudo -u www-data /var/www/nextcloud/occ config:app:set --value=“64 128 1024” previewgenerator widthSizes

Responce was
Config value widthSizes for app previewgenerator set to 64 128 1024

sudo -u www-data /var/www/nextcloud/occ config:app:set --value=“64 256 1024” previewgenerator heightSizes

Responce was
Config value heightSizes for app previewgenerator set to 64 256 1024

Here is after deleting all my data and emptying the trash bin:

ncadmin@cloud:~$ sudo du -ah --max-depth=1  /mnt/ncdata | sort -hr
160M    /mnt/ncdata
129M    /mnt/ncdata/appdata_ocm2710gjvgh
11M     /mnt/ncdata/ncadmin
11M     /mnt/ncdata/jtorres
11M     /mnt/ncdata/datorres
213K    /mnt/ncdata/files_external
76K     /mnt/ncdata/dtorres
65K     /mnt/ncdata/nextcloud.log
4.5K    /mnt/ncdata/updater.log
4.5K    /mnt/ncdata/.htaccess
512     /mnt/ncdata/updater-ocm2710gjvgh
512     /mnt/ncdata/.ocdata
512     /mnt/ncdata/index.html

Another view is:

ncadmin@cloud:~$ df -Th
Filesystem                     Type      Size  Used Avail Use% Mounted on
udev                           devtmpfs  1.9G     0  1.9G   0% /dev
tmpfs                          tmpfs     393M  1.2M  391M   1% /run
/dev/mapper/nextcloud--vg-root ext4       39G  8.7G   28G  25% /
tmpfs                          tmpfs     2.0G  8.0K  2.0G   1% /dev/shm
tmpfs                          tmpfs     5.0M     0  5.0M   0% /run/lock
tmpfs                          tmpfs     2.0G     0  2.0G   0% /sys/fs/cgroup
ncdata                         zfs       641M  161M  481M  25% /mnt/ncdata
tmpfs                          tmpfs     393M     0  393M   0% /run/user/1000

Space seems to no longer be an issue. I still problems with my Cert but i’ll open a new thread on that.

Thanks for all the help!!

1 Like
#23

@ gas85

Sorry, seems like I spoke to soon. My disk usage is now:

ncadmin@cloud:~$ df -Th
Filesystem                     Type      Size  Used Avail Use% Mounted on
udev                           devtmpfs  1.9G     0  1.9G   0% /dev
tmpfs                          tmpfs     393M  1.3M  391M   1% /run
/dev/mapper/nextcloud--vg-root ext4       39G  9.4G   27G  26% /
tmpfs                          tmpfs     2.0G   72K  2.0G   1% /dev/shm
tmpfs                          tmpfs     5.0M     0  5.0M   0% /run/lock
tmpfs                          tmpfs     2.0G     0  2.0G   0% /sys/fs/cgroup
ncdata                         zfs        33M   33M     0 100% /mnt/ncdata
tmpfs                          tmpfs     393M     0  393M   0% /run/user/1000

ncadmin@cloud:~$ sudo du -ah --max-depth=1  /mnt/ncdata | sort -hr

33M     /mnt/ncdata
11M     /mnt/ncdata/ncadmin
11M     /mnt/ncdata/datorres
5.6M    /mnt/ncdata/appdata_ocm2710gjvgh
5.5M    /mnt/ncdata/jtorres
733K    /mnt/ncdata/nextcloud.log
177K    /mnt/ncdata/files_external
65K     /mnt/ncdata/dtorres
4.5K    /mnt/ncdata/updater.log
4.5K    /mnt/ncdata/.htaccess
512     /mnt/ncdata/updater-ocm2710gjvgh
512     /mnt/ncdata/.ocdata
512     /mnt/ncdata/index.html

Compare the df -Th with one from a few weeks ago:

ncadmin@cloud:~$ df -Th
Filesystem                     Type      Size  Used Avail Use% Mounted on
udev                           devtmpfs  1.9G     0  1.9G   0% /dev
tmpfs                          tmpfs     393M  1.3M  391M   1% /run
/dev/mapper/nextcloud--vg-root ext4       39G  8.9G   28G  25% /
tmpfs                          tmpfs     2.0G  8.0K  2.0G   1% /dev/shm
tmpfs                          tmpfs     5.0M     0  5.0M   0% /run/lock
tmpfs                          tmpfs     2.0G     0  2.0G   0% /sys/fs/cgroup
ncdata                         zfs        24G   24G  497M  98% /mnt/ncdata
tmpfs                          tmpfs     393M     0  393M   0% /run/user/1000

I can no longer login with out getting the, Internal Server Error, message. How is it possible the ncdata partition is shrinking?

Any guidance would be greatly appreciated. Thanks again!

#24

Using the preview generator app will fill your disk if not setup properly. Hence there is a script for it in the VM that can be found here.

#25

@enoch85 Thanks for the script. I ran it till successful but this did not address my issue. As seen below the appdata folder is a good size. I can no longer login with out getting the, Internal Server Error, message. How is it possible the ncdata partition is shrinking?

ncadmin@cloud:~$ df -Th
Filesystem Type Size Used Avail Use% Mounted on
udev devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs tmpfs 393M 1.3M 391M 1% /run
/dev/mapper/nextcloud–vg-root ext4 39G 9.4G 27G 26% /
tmpfs tmpfs 2.0G 72K 2.0G 1% /dev/shm
tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
ncdata zfs 33M 33M 0 100% /mnt/ncdata
tmpfs tmpfs 393M 0 393M 0% /run/user/1000

ncadmin@cloud:~$ sudo du -ah --max-depth=1 /mnt/ncdata | sort -hr

33M /mnt/ncdata
11M /mnt/ncdata/ncadmin
11M /mnt/ncdata/datorres
5.6M /mnt/ncdata/appdata_ocm2710gjvgh
5.5M /mnt/ncdata/jtorres
733K /mnt/ncdata/nextcloud.log
177K /mnt/ncdata/files_external
65K /mnt/ncdata/dtorres
4.5K /mnt/ncdata/updater.log
4.5K /mnt/ncdata/.htaccess
512 /mnt/ncdata/updater-ocm2710gjvgh
512 /mnt/ncdata/.ocdata
512 /mnt/ncdata/index.html

Compare the df -Th with one from a few weeks ago:

ncadmin@cloud:~$ df -Th

Filesystem Type Size Used Avail Use% Mounted on
udev devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs tmpfs 393M 1.3M 391M 1% /run
/dev/mapper/nextcloud–vg-root ext4 39G 8.9G 28G 25% /
tmpfs tmpfs 2.0G 8.0K 2.0G 1% /dev/shm
tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
ncdata zfs 24G 24G 497M 98% /mnt/ncdata
tmpfs tmpfs 393M 0 393M 0% /run/user/1000

Any guidance would be greatly appreciated. Thanks again!

#26

From 24 GB to 33 Mb? Do no know :face_with_raised_eyebrow:
Could be that you have Hardware issue? Did you lost all your data in NC?

#27

Hi Georgiy, I have become so frustrated with storage issues I have moved all my data to a Windows server. I am now using NC external disks and sync data directly. NC is now solely as a pass thru but now that is not working either. I’m shocked that so much work would be needed just to keep it stable. Admittedly I am not strong a Linux person.

This is a Hyper-V Virtual Machine, so no hardware issues are possible. The underlining host is 100%. I am now focused on the " Internal Server Error". I have reviewed many postings of the same error but can’t find a solution. Any logs to look that might help?

Thanks again!

#28

I have’t used Hyper-V, but NC is pretty simple - it is just an application with DB access and you can map DATA folder to whatever place via config or mount. Also you can added external storage as local folder if you need. Basically you need only to bother about DB access, local storage access and make server (with Apache2 or ngnix) that will serve your https connections, additionally added you Letsencrypt cert and it is done.
It seems that something wrong with you virtual machine, but as I did not used Hyper-V, I can’t help you ad-hoc :frowning:
Try to open a ticket in GitHub with logs and all relevant data.


or for Docker:

#29