Nc-backup - free space check failed (quick question)

Edson_Rodrigues · November 23, 2021, 4:26pm

I have a NCP instance on a 500GB SSD which is now 90% capacity so I attached a USB 1TB SSD to do a full backup and move the instance to other computer and disk. However, it fails:
.

check free space...
free space check failed. Need 405651974018 Bytes
Maintenance mode disabled
Done. Press any key...

.

root@nextcloudpi:/media# df -H
Filesystem      Size  Used Avail Use% Mounted on
udev            8.4G     0  8.4G   0% /dev
tmpfs           1.7G   27M  1.7G   2% /run
/dev/sda1       491G  491G     0 100% /
tmpfs           8.4G     0  8.4G   0% /dev/shm
tmpfs           5.3M     0  5.3M   0% /run/lock
tmpfs           8.4G     0  8.4G   0% /sys/fs/cgroup
tmpfs           1.7G     0  1.7G   0% /run/user/1000
tmpfs           1.7G     0  1.7G   0% /run/user/0
/dev/sdb1       1.1T   21G  978G   3% /media/usb

.
.

the /dev/sdb1 is mounted on /media/usb and it is where I want to store the backup:

Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 840
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x108c5f70

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1  *         2048 974772223 974770176 464.8G 83 Linux
/dev/sda2       974774270 976771071   1996802   975M  5 Extended
/dev/sda5       974774272 976771071   1996800   975M 82 Linux swap / Solaris



Disk /dev/sdb: 931.4 GiB, 1000123400192 bytes, 1953366016 sectors
Disk model:
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
Disklabel type: gpt
Disk identifier: 7830567C-2150-5042-97EC-FF158E645FCD

Device     Start        End    Sectors   Size Type
/dev/sdb1  65536 1953365982 1953300447 931.4G Linux filesystem

.

i.e.: if I have a NCP instance covering 200GB do I need 200GB free to back it up? even if the backup is in another HDD?

Anyone can help me on possible solutions to this?

Thanks,
ER

sven1234 · November 23, 2021, 8:53pm

Do you actually have a partition on that disk? Should it not be /dev/sdb1?

Edson_Rodrigues · November 23, 2021, 8:59pm

Hi @sven1234

Yes I do, bad copy/paste have a partition and formated in btrfs. I am now testing to copy the /var/www/nextcloud/ to the disk via rsync.

I will edit the original post.

Thanks,
ER

Edson_Rodrigues · November 24, 2021, 10:16am

I backed up my data with rsync to external USB SSD just to make sure everything was “OK”.

When it was done, I tried again :

root@nextcloudpi:/# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 465.8G  0 disk
├─sda1   8:1    0 464.8G  0 part /
├─sda2   8:2    0     1K  0 part
└─sda5   8:5    0   975M  0 part [SWAP]
sdb      8:16   0 931.4G  0 disk
└─sdb1   8:17   0 931.4G  0 part /media/usb

root@nextcloudpi:/# btrfs filesystem show
Label: none  uuid: 17a4adbc-c718-44ca-accb-d208ed1c66e3
        Total devices 1 FS bytes used 378.10GiB
        devid    1 size 931.41GiB used 382.02GiB path /dev/sdb1

root@nextcloudpi:/# btrfs filesystem usage /media/usb
Overall:
    Device size:                 931.41GiB
    Device allocated:            382.02GiB
    Device unallocated:          549.39GiB
    Device missing:                  0.00B
    Used:                        378.86GiB
    Free (estimated):            550.05GiB      (min: 275.35GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              422.97MiB      (used: 0.00B)

Data,single: Size:378.00GiB, Used:377.34GiB
   /dev/sdb1     378.00GiB

Metadata,DUP: Size:2.00GiB, Used:776.11MiB
   /dev/sdb1       4.00GiB

System,DUP: Size:8.00MiB, Used:64.00KiB
   /dev/sdb1      16.00MiB

Unallocated:
   /dev/sdb1     549.39GiB

I assume it is safe to say that /dev/sdb1 is not mounted properly on /media/usb and using btrfs

Now seems it is working

Running nc-backup
check free space...
Maintenance mode enabled
backup database...
backup files...

Now it should take some time:

… time to make a cup of coffee

devnull · November 24, 2021, 10:19am

Sorry i do not read all entrys. But perhaps you waste some space with e.g. nextcloud upgrade - backups in data/updater-***/backups or on other positions.

Please install ncdu and find big directorys.

ncdu /path/to/your/nextcloud
ncdu /media/usb

100% full disk can crash your system.
Normally non-root-user like www-data can not fill it to 100%.

Edson_Rodrigues · November 24, 2021, 10:38am

Yesterday night I had around 20% free and this morning it was gone. I think a cron job related with the photo/video preview gen started and used that free space.

And you are right, once it reached 100%, my instance was gone… but I manage to get some free space deleting some things related with the preview generator… or so I hope

Thanks for the tip on using “ncdu”, much easier than du/df

I have 2 users, seems that they are the ones taking the majority of the space so this should be as expected.

Thanks,
ER

devnull · November 24, 2021, 11:25am

You can set a quota in nextcloud. It is useful if you are not the only user.

devnull · November 24, 2021, 11:27am

I think you do not use the /media/usb for storage. Is this for you correct?
Look your ncdu-screenshot (/var/www/nextcloud/data not /media/usb)

Please check this.

Edson_Rodrigues · November 24, 2021, 12:07pm

Yes, I know I can set quote, in this case for both users the quote is/should be “unlimited” and it is not a problem. I know I am suddenly reaching full disk capacity thus this exercise of backup/move to other instance.

Regarding the

> I think you do not use the /media/usb for storage. Is this for you correct?

Indeed, /media/usb is the mount point for /dev/sdb1 which is the 1GB SSD disk where I am backing up my NextCloudPi

Edson_Rodrigues · November 24, 2021, 12:13pm

OK, back of the instance + data + compression was OK:

Running nc-backup
check free space...
Maintenance mode enabled
backup database...
backup files...
backup /media/usb/nextcloud-bkp_20211124_1637746823.tar.gz generated
Maintenance mode disabled
Done. Press any key...

Here is my backup done on the external 1TB SSD mount point /media/usb:

-rw-r----- 1 root     www-data 373G Nov 24 11:11 nextcloud-bkp_20211124_1637746823.tar.gz

Now I will try to make the exact same backup but on a external USB 500GB drive to see if all goes OK.

Edson_Rodrigues · November 24, 2021, 10:20pm

Backup works fine, but restore fails with:

Running nc-restore
check free space...
free space check failed. Need 393707054 KB in /media/usb500gb/ncp-restore.JFczsV
Cleanup...
Done. Press any key...

Looks like the tar.gz will be unpacked complete then the files copied? Looks like it by looking at the processes running and this means that I will always need 2x the free space that the backup takes so that it hold the .tar.gz + the unpacked files.

My 373GB tar.gz is requires ~393GB free to be unpacked. This means that although I was able to create such backup in a 500GB SSD, I will be unable to restore from it.

Can someone confirm that this is how it works?

Thanks,
ER.

Simon2018 · November 24, 2022, 11:16am

I have the same issue. How did you solve it? copy to larger HDD?
Is there a better possibility to backup?

Edson_Rodrigues · November 24, 2022, 11:51am

I could not get a reply on this thread, so I assume now that I need 2x the free space, I think i put up a workaround where I moved the backup to a much bigger external disk, created new NCP install where I had 2-3x the free space then i was able to restore it…

schoetju · November 24, 2022, 7:27pm

Other option could be to decompress the tar.gz file manually to a different drive (so that you have a uncompressed .tar file).
However, that’ll need some linux-know-how.

Easiest option: Go for a bigger external disk