Disk full after trying to backup

ℹ️ Support
1

Hi everybody !

Nextcloud version (eg, 20.0.5): 23
Operating system and version (eg, Ubuntu 20.04): Ubuntu 16.04.4
Installer : Snap
Is this the first time you’ve seen this error? (Y/N): Yes

The issue you are facing:

I have a Nextcloud instance installed with snap on a virtual machine with only Nextcloud installed and 1.8 To of disk space.
I tried to transfert my personal Nextcloud instance to a managed one because I did not want to continue the maintenance of my Nextcloud.

I tried to use this functionnality to do the tranfert : Beta of peer-to-peer Nextcloud Backup app now available for testing! - Nextcloud
The server was in a maintenance mode during the transfert, but, after a day, I have this error message :

Erreur
Impossible d’écrire dans le répertoire "apps"

Ce problème est généralement résolu en donnant au serveur web un accès en écriture au répertoire des applications ou en désactivant le magasin d'applications dans le fichier de configuration.

I let it like this during months because I did not have time to fix it :sweat_smile:
Now, I am trying to solve the problem.
I use SSH to connect myself to the server, and I see is my disk completely full :

root@ns379063:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            2.0G     0  2.0G   0% /dev
tmpfs           394M   48M  347M  13% /run
/dev/sda2       1.8T  1.8T     0 100% /
tmpfs           2.0G     0  2.0G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/sda1       487M   58M  400M  13% /boot
/dev/loop2       56M   56M     0 100% /snap/core18/2284
/dev/loop5      111M  111M     0 100% /snap/core/12725
/dev/loop4      239M  239M     0 100% /snap/nextcloud/29844
/dev/loop1      111M  111M     0 100% /snap/core/12821
/dev/loop6      248M  248M     0 100% /snap/nextcloud/29929
/dev/loop0       56M   56M     0 100% /snap/core18/2344

sda2 no have space anymore.
I guess it is the backup soft wich try to create the backup but failed because of disk space.

Now, I am trying to make some free space to enable my Nextcloud again, but all my tests failed because of the leak of memory.

root@ns379063:/# sudo apt-get autoremove
Reading package lists... Error!
E: Write error - write (28: No space left on device)
E: IO Error saving source cache
E: The package lists or status file could not be parsed or opened.

root@ns379063:/# sudo du -sh /var/cache/apt 
99M	/var/cache/apt
root@ns379063:/# sudo apt-get autoclean
Reading package lists... Error!
E: Write error - write (28: No space left on device)
E: IO Error saving source cache
E: The package lists or status file could not be parsed or opened.

root@ns379063:~# nextcloud.occ trashbin:cleanup --all-users
error: cannot communicate with server: timeout exceeded while waiting for response
error: cannot communicate with server: timeout exceeded while waiting for response
error: cannot communicate with server: timeout exceeded while waiting for response
error: cannot communicate with server: timeout exceeded while waiting for response
error: cannot communicate with server: timeout exceeded while waiting for response
error: cannot communicate with server: timeout exceeded while waiting for response
Nextcloud is not installed - only a limited number of commands are available
Cannot write into "apps" directory
This can usually be fixed by giving the webserver write access to the apps directory or disabling the App Store in the config file.

An unhandled exception has been thrown:
Exception: Environment not properly prepared. in /snap/nextcloud/29929/htdocs/lib/private/Console/Application.php:164
Stack trace:
#0 /snap/nextcloud/29929/htdocs/console.php(98): OC\Console\Application->loadCommands()
#1 /snap/nextcloud/29929/htdocs/occ(11): require_once('...')

I found the nextcloud data folder, but I don’t really want to just rm some files, because I am wondering about the mySQL database after, will it continue to work well ?

The output of your Nextcloud log in Admin > Logging:

Can't access

The output of your config.php file in /var/snap/nextcloud/current/nextcloud/config/ (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'myInstanceId, do you need it ?',
);

I don’t understand exactly wich logs you want, so I send what I found in /var/snap/nextcloud/current/logs :

apache_error.log

[Sun Aug 28 15:14:44.263995 2022] [ssl:warn] [pid 1488:tid 140346123871424] AH01873: Init: Session Cache is not configured [hint: SSLSessionCache]
[Sun Aug 28 15:14:44.264533 2022] [lbmethod_heartbeat:notice] [pid 1488:tid 140346123871424] AH02282: No slotmem from mod_heartmonitor
[Sun Aug 28 15:14:44.268070 2022] [mpm_event:notice] [pid 1488:tid 140346123871424] AH00489: Apache/2.4.52 (Unix) OpenSSL/1.1.1 configured -- resuming normal operations
[Sun Aug 28 15:14:44.268179 2022] [core:notice] [pid 1488:tid 140346123871424] AH00094: Command line: 'httpd -d /snap/nextcloud/29929 -D FOREGROUND'
[Sun Aug 28 15:14:44.272649 2022] [unixd:alert] [pid 1490:tid 140346123871424] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive
[Sun Aug 28 15:14:44.273046 2022] [unixd:alert] [pid 1491:tid 140346123871424] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive
[Sun Aug 28 15:14:44.275918 2022] [unixd:alert] [pid 1489:tid 140346123871424] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive
[Sun Aug 28 15:14:48.887575 2022] [log_config:warn] [pid 1490:tid 140345941669632] (28)No space left on device: [client 185.104.185.102:44066] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:14:48.926905 2022] [log_config:warn] [pid 1489:tid 140345950062336] (28)No space left on device: [client 185.104.185.102:44068] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:14:48.952811 2022] [log_config:warn] [pid 1490:tid 140345950062336] (28)No space left on device: [client 185.104.185.102:44072] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:14:48.976354 2022] [log_config:warn] [pid 1489:tid 140345941669632] (28)No space left on device: [client 185.104.185.102:44068] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:14:48.995645 2022] [log_config:warn] [pid 1489:tid 140345891313408] (28)No space left on device: [client 185.104.185.102:44074] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:14:49.059287 2022] [log_config:warn] [pid 1491:tid 140345941669632] (28)No space left on device: [client 185.104.185.102:44070] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:14:49.208212 2022] [log_config:warn] [pid 1491:tid 140345933276928] (28)No space left on device: [client 185.104.185.102:44070] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:29:49.315033 2022] [log_config:warn] [pid 1489:tid 140345958455040] (28)No space left on device: [client 185.104.185.102:51406] AH00646: Error writing to /var/snap/nextcloud/29929/logs/apache_access.log
[Sun Aug 28 15:32:12.405610 2022] [unixd:alert] [pid 3326:tid 140346123871424] AH02155: getpwuid: couldn't determine user name from uid 4294967295, you probably need to modify the User directive

mysql_errors.log, php_errors.log, php-fpm_errors.log and redis.log are empty
Do you need apache_access.log ? It looks like having ip data and other personal stuff.

I hope to give you enough data, if you need more you can ask me.
Sorry for my bad english, I am french and I did not write since a while :laughing:

Thank you and have a good day,

Julien

2

Sorry i do not use Snap-installation.

But on normal installations there are backups at
/path/to/nextcloud/data/updater-***/backups

Also you can install the linux program ncdu and list the paths by size. Then you can surf in this text program to the biggest folders.

Please note that if your hard drive is full, software may be installed incorrectly or incompletely. This may have broken your nextcloud installation.

3

Hi @devnull !

Thank you for your answer.
It look likes you find the good thing, I found the backup directory, I guess, here :
/var/snap/nextcloud/common/nextcloud/data/appdata_ocf8ek6msbgl/backup

And the space occupied look likes logical :

root@ns:/var/snap/nextcloud/common/nextcloud/data/appdata_ocf8ek6msbgl/backup# du -sh 20220401143907-full-CiD5GrH5I4oeHOr/
898G	20220401143907-full-CiD5GrH5I4oeHOr/

I think it is the backup created by the soft.
Does anybody can confirm me that please ? I am pretty afraid to delete it actually :sweat_smile:

4

I’d be more afraid that in the end the restore won’t work. You should also try this out as part of an emergency precaution. I think the most problems tend to occur during restore.

5

Hi @devnull !

Few days after, I download the entire file in a local external hard disk, as a backup, and I deleted the folder.
Now, my VM is breathing :slight_smile:

So, I did an apt-get update and an apt-get upgrade to have a clean environment.
I see just now that my Nextcloud upgraded itself with the apt-get upgrade (actually I did not want, but it is to late ><)

# snap list
Name       Version      Rev    Tracking       Publisher   Notes
core       16-2.56.2    13425  latest/stable  canonical✓  core
core18     20220830     2560   latest/stable  canonical✓  base
nextcloud  24.0.3snap1  31222  latest/stable  nextcloud✓  -

And now, when I trying to have more information about my Nextcloud :

# nextcloud.occ status
Nextcloud is not installed - only a limited number of commands are available
Cannot write into "apps" directory.
This can usually be fixed by giving the web server write access to the apps directory or disabling the App Store in the config file.

An unhandled exception has been thrown:
Exception: Environment not properly prepared. in /snap/nextcloud/31222/htdocs/lib/private/Console/Application.php:164
Stack trace:
#0 /snap/nextcloud/31222/htdocs/console.php(99): OC\Console\Application->loadCommands()
#1 /snap/nextcloud/31222/htdocs/occ(11): require_once('...')

It’s weird, why do I have Nextcloud is not installed - only a limited number of commands are available? :thinking:

Should I post an issue on github ?

Thank you for your help.

6

Unfortunately, I am not familiar with Snap. It may be quite easy to install. It may also work well as long as it works. But when it comes to problems, unfortunately my normal linux experience is not enough. I think Snap knowledge is necessary here. Maybe someone can help.

Maybe your Ubuntu version is to old. Post the actual settings and more errors, logs and configs. If you want to install new maybe without Snap. Read this.

7

Hi @devnull

Thank you for your message.
I see that some people have the same issue with exactly the same process than me on github :

So, I will close the issue here and continue there to centralyze the data about this bug :wink:

Thank you very well for your help :slight_smile: