Nextcloud Box Ubunto Appliance Crashing on large data transfer

Hi

I have Nextcloud Ubuntu Appliance on my RPi3 Nextcloud Box which has an external hard disk, and the 3.1.2 Client on Manjaro Arch Linux

All is working perfectly, except for when I change a lot of files on my Laptop (e.g. 30GB) the client starts syncing OK, but then the RPi crashes every hour or so.

At least I think it is crashing as the route to host is lost for SSH from Laptop.

What are the relevant logs that I should look at to see what is happening please?
(well so that I can post on here to ask someone how to work it out :grinning: )

Cheers

PS I have used the below before after physically connecting the box hard disk to the laptop and copy the files over (with cp -pr ) but I am now trying to get it to work via the client for large data exchanges
sudo nextcloud.occ files:scan --path="<Nextcloud user_id>/files/<name given to disk in Nextcloud>/<Directory of interest on external disk>"

OK, now I have a bigger problem - it seemed to be working better so I left it overnight but this morning I have the following error:

500 internal server error to PUT ...

I ended up reinstalling everything again

I think I might be getting closer to understanding this?

For future readers …

I think (hope :grinning: ) I have solved this by just getting a bigger (124GB) SD card for my RPi:

I was getting this problem on large data transfers (for example when Shotwell rewrote the metadata to all my many photos due to me forgetting to mount the external disk)

I suspected that:

  • During the data transfer the SD card of the RPi was getting full.
  • Since this ocurred as the data was being transferred making more space (see my methods for this at end of this post) on the SD card after this was too late to be any help

The SD card I was using was an 8GB one that was way bigger than the Core Appliance needed, but I thought that perhaps a bigger one would solve my problem

So I got a 124GB SD card (around €15) and cloned the (working) SD disk to this

Just now, I had the Shotwell problem again, but the RPi coped with the large data transfer :smiley:

Woohoo!

ME@localhost:~$ df -H
Filesystem      Size  Used Avail Use% Mounted on
udev            404M     0  404M   0% /dev
tmpfs            96M   11M   85M  12% /run
/dev/mmcblk0p2  126G  8.5G  111G   8% /writable
/dev/loop0       52M   52M     0 100% /
/dev/loop1      195M  195M     0 100% /lib/modules
tmpfs           476M  4.1k  476M   1% /etc/fstab
tmpfs           476M     0  476M   0% /dev/shm
tmpfs           5.3M     0  5.3M   0% /run/lock
tmpfs           476M     0  476M   0% /sys/fs/cgroup
tmpfs           476M     0  476M   0% /var/lib/sudo
tmpfs           476M     0  476M   0% /mnt
tmpfs           476M     0  476M   0% /media
tmpfs           476M  164k  475M   1% /tmp
/dev/loop2      325M  325M     0 100% /snap/nextcloud/28552
/dev/loop3       12M   12M     0 100% /snap/pi/74
/dev/loop4      320M  320M     0 100% /snap/nextcloud/28579
/dev/loop5       30M   30M     0 100% /snap/snapd/13269
/dev/loop6       12M   12M     0 100% /snap/pi/98
/dev/loop7      320M  320M     0 100% /snap/nextcloud/28592
/dev/loop8       30M   30M     0 100% /snap/snapd/13171
/dev/loop11      30M   30M     0 100% /snap/snapd/13643
/dev/loop10     194M  194M     0 100% /snap/pi-kernel/345
/dev/loop9       52M   52M     0 100% /snap/core18/1888
/dev/loop12     201M  201M     0 100% /snap/pi-kernel/185
/dev/sda1       984G  584G  351G  63% /media/Work
/dev/mmcblk0p1  265M  128M  137M  49% /boot/uboot
tmpfs            96M     0   96M   0% /run/user/1000

The /dev/mmcblk0p2 126G 8.5G 111G 8% /writable line seems to show that if the card was only the previous 8GB one then I would have had a problem

I then ran

sudo nextcloud.occ versions:cleanup

which got it down to

/dev/mmcblk0p2  126G  3.5G  116G   3% /writable

But

sudo -i
cd /var/snap/nextcloud/common/nextcloud/data/Leigh/files\_versions
rm -R *

and

sudo nextcloud.occ trashbin:cleanup --all-users

Didnt make any further difference, i.e. it was still at

/dev/mmcblk0p2  126G  3.5G  116G   3% /writable

Update 09/11/21
Alll has been working perfectly with the new card

I just changed the tags on a lot of photos with Shotwell, precipitating a 3 GB upload to the Nextcloud Box and all went well (which would not have happened before I got the new card)

However, df -H now gives 12G used on the card:

/dev/mmcblk0p2 126G 12G 108G 11% /writable

So the card has gradually been filling up

sudo nextcloud.occ versions:cleanup
sudo nextcloud.occ trashbin:cleanup --all-users

got it back down to 3.6 G (about half with each command)

/dev/mmcblk0p2  126G  3.6G  116G   3% /writable

But shouldnt it automatically clean out these things?

This may be ‘Famous last Words’ but …

Since taking @kyrofa 's advice and changing from appliance to Ubuntu Server then simply installing Nextcloud snappy, all seems much better with no problems

Also avoiding core means its easier to set up with external HDD automount, etc

Famous last words indeed
I’m now back with the same old problem