Is It okey to telete trash only from hosting filesystem?

Dusan_Krehel · December 15, 2021, 7:32pm

Hi.

The users have more files in Trash. I want to delete those files in bulk. Can I telete the trashes only from hosting filesystem? Then will data Nextcloud database okey?

D. K.

mainland · December 16, 2021, 12:29am

The best way to clean up deleted files is to automate the function in your config file. Simply insert the following line into your config:

'trashbin_retention_obligation' => 'auto, 14',

This will delete all trashed files when the bin is full or when files have been in the bin for the specified number of days; whichever comes first.

You will have to restart your webserver to have the changes take place.

gas85 · December 16, 2021, 9:38am

This will cause issues in DB, use command for this, as per docu https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html#trashbin

sudo -u www-data php occ trashbin:cleanup

You can even do this for particular users.

mainland · December 16, 2021, 2:28pm

Both approaches do the same thing but the ‘trashbin_retention_obligation’ automates it across all uses.

I run three production Nextcloud servers and found that users were moving GBs of archived data onto NC and then deleting it which caused the storage to increase significantly. This is the only way I can keep it under control.

gas85 · December 16, 2021, 2:56pm

They should not goes over 50% of given quota with trash-bin…

Ok if we talk about approach, then from my point of view setting 'auto, 14' is quite rough to users, I would rather set it to default setting 'auto', this is more user friendly and “keeps files and folders in the trash bin for 30 days and automatically deletes anytime after that if space is needed (note: files may not be deleted if space is not needed).”

Or more even user friendly will be '14, auto' to keep files 14 days and then delete if needed.

mainland · December 16, 2021, 4:04pm

I agree that if the NC server has storage room there is no reason to automatically delete files. But this approach ignores the trashbin’s impact on the size of backups.

My issue is that one of the corporate servers I maintain has 240GB of data on it and taking rolling backups requires at least 4X that much storage. If I don’t limit the the trashbin then it will consume the trashbin size X4 on my backup media. I am already carrying 1TB of backup data from that one server so any additional GBs in the trashbin is meaningful.

Another fundamental organizational question is “what is the trashbin for?” It should not be just another folder of last resort to hold deleted files that may be of use in 6 months. If it is trash it should be disposed of.

gas85 · December 17, 2021, 7:36am

I use restic to perform backups and my daily backup size is even smaller (about 500 GB) than whole data folder (about 700 GB), because of compression and hardlinks to the same and unchanged files. Check out, there few posts about it, e.g. this one Question for backup and restore - #3 by Reiner_Nippes

Also I find out that there is a lot of redundant data:

github.com/nextcloud/server

Use hardlinks instead of copy when possible

opened 09:47AM - 10 Feb 21 UTC

closed 12:02PM - 09 Aug 21 UTC

GAS85

enhancement 0. Needs triage Nice to have

### How to use GitHub * Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are interested into the same feature. * Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue. * Subscribe to receive notifications on status change and new comments. **Is your feature request related to a problem? Please describe.** When user copy file to different folder in local Storage, then it is copied and not hard linked, that will increase space consumption by the cloud Server. Those are the same files, but different Inodes. ``` ls -lai Da*.txt Shared/Datenschutzerklärung.txt 134284604 -rw-r--r-- 1 www-data www-data 4783 Oct 7 2019 Datenschutzerklärung.txt 134352533 -rw-r--r-- 1 www-data www-data 4783 Feb 10 10:30 Shared/Datenschutzerklärung.txt md5sum Da*.txt Shared/Datenschutzerklärung.txt 3712399a1ddddc19b088bd805407b361 Datenschutzerklärung.txt 3712399a1ddddc19b088bd805407b361 Shared/Datenschutzerklärung.txt ``` **Describe the solution you'd like** - First try to create hardlink to the file in new location on local storage, only if it is fails then try to use copy command. - Make Hardlinks creation optional, e.g. not relevant for S3 Storage on AWS. **Describe alternatives you've considered** Currently I use `rdfind -makehardlinks true` to find same files and reduce space usage. This will also use other users folder, e.g. common Family Videos from different users will be hardlinked. **Additional context** Similar behavior implemented e.g. in Sonarr. https://github.com/Sonarr/Sonarr/wiki/Completed-Download-Handling You can setup if Hardlink should be used instead of copy and If the hardlink creation fails, Sonarr will fall back to the default behavior and copy the file. Also related to #19948

And I can reduce used space with rdfind -makehardlinks true, or this script to exclude e.g. appdata_* from it:

github.com

GAS85/nextcloud_scripts/blob/master/nextcloud-hardlink-duplicates.sh

#!/bin/bash
 
# By Georgiy Sitnikov.
#
# AS-IS without any warranty

NextCloudPath=/var/www/nextcloud
Command=/var/www/nextcloud/occ

# Do not touch (true) appdata_XXXXXX directory, e.g. previews and system/app files.
# If you set false, appdata and previews will be also evaluated
appdata=true

###

LOCKFILE=/tmp/nextcloud-hardlink-duplicates.tmp

CentralConfigFile="/etc/nextcloud-scripts-config.conf"

if [ -f "$CentralConfigFile" ]; then

This file has been truncated. show original

mainland · December 17, 2021, 11:28am

Thanks for the info. I’ll look at your suggestions and maybe make some changes to my backup routine.

I use a simple sudo tar -czvf' for backing up the various servers. It’s slow and simple but does the job. It also makes extraction easy when rebuilding.

Dusan_Krehel · December 17, 2021, 4:28pm

Reboot of server maybe unreal, because the web page is on multihosting of web-provider (I’m not technical admin of web-server).
Auto doesn’t must work (real disk size > hosting web-page dir).
The shell access doesn’t work – web-page dir is over NFS (another absolute path in shell and real web (as in config file of Nextcloud) – I don’t can set the right parameter for PATH of web-page dir.