Nextcloud AIO freezes, have to manually stop all containers to get it responsive

[beppi]

Hey guys,

First time setting up a Nextcloud AIO server, so be please be gentle!

Server: Ubuntu 22.04
Nextcloud AIO: 6.2.1
Server: Dell XPS 15 (~2015)
CPU: Intel core i7
Memory: 16 GB
Storage: 1 TB HDD

I’ve been migrating all my stuff to a new laptop/server (just a calibre server), and thought while I’m here I’ll try Nextcloud! It seems very promising, however, I have some questions/issues:

1. I’ve been uploading photos/vids around 30 at a time, from google photos. Sometimes though, the server will just slow down to a crawl, or freeze, and after a while I’ll get a timeout error. I can’t get access to the containers until I manually ssh into the server and docker compose down/docker stop all of the containers. HOWEVER, as annoying as this is, the most recent restart hasn’t prevented it from freezing. Running free in the server shows that there is 15GB total, 6GB used, 200MB free??? and 7 GB available. Do I need to allow docker to use more than 6GB? So I dont just have 7GB sitting there waiting? The server isn’t doing anything at the moment so I don’t know how it could use 6GB anyway…

2. How do I actually stop the Nextcloud-AIO? I’ve been using docker compose down, but it doesn’t stop the other containers, just the ones defined in my compose.yml. So I’ve been using docker stop on all of the others, but surely there’s a better way.

Please help!! I really want this to work, I love how I can store my contacts, calendar and photos on my home server, and I just can’t seem to figure these issues out myself.

Current process that causes this error:
Open Nextcloud Web > Go to files > Upload some files > Click to view one of the new files (Web or Mobile) > Doesn’t load, Nextcloud starts lagging, and RAM starts ramping up (memory leak?)

Output from docker logs nextcloud-aio-mastercontainer:

Trying to fix docker.sock permissions internally…
Creating docker group internally with id 137
e[0;92mInitial startup of Nextcloud All-in-One complete!
You should be able to open the Nextcloud AIO Interface now on port 8080 of this server!
E.g. https://internal.ip.of.this.server:8080

If your server has port 80 and 8443 open and you point a domain to your server, you can get a valid certificate automatically by opening the Nextcloud AIO Interface via:
https://your-domain-that-points-to-this-server.tld:8443e[0m
{“level”:“info”,“ts”:1688886448.64077,“msg”:“using provided configuration”,“config_file”:“/Caddyfile”,“config_adapter”:“”}
{“level”:“info”,“ts”:1688886449.11291,“msg”:“failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size for details.”}
[Sun Jul 09 07:07:35.790104 2023] [mpm_event:notice] [pid 147:tid 140670822898504] AH00489: Apache/2.4.57 (Unix) OpenSSL/3.1.1 configured – resuming normal operations
[Sun Jul 09 07:07:35.790138 2023] [core:notice] [pid 147:tid 140670822898504] AH00094: Command line: ‘httpd -D FOREGROUND’
[09-Jul-2023 07:07:37] NOTICE: fpm is running, pid 151
[09-Jul-2023 07:07:37] NOTICE: ready to handle connections
Restarting cron.sh because daily backup time was set, changed or unset.
++ head -1 /mnt/docker-aio-config/data/daily_backup_time

• BACKUP_TIME=20:00
• export BACKUP_TIME
• export DAILY_BACKUP=1
• DAILY_BACKUP=1
  ++ sed -n 2p /mnt/docker-aio-config/data/daily_backup_time
• ‘[’ ‘’ ‘!=’ automaticUpdatesAreNotEnabled ‘]’
• export AUTOMATIC_UPDATES=1
• AUTOMATIC_UPDATES=1
• set +x
  NOTICE: PHP message: 404 Not Found
  Type: Slim\Exception\HttpNotFoundException

4
Message: Not found.
File: /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/RoutingMiddleware.php
Line: 76
Trace: #0 /var/www/docker-aio/php/vendor/slim/slim/Slim/Routing/RouteRunner.php(56): Slim\Middleware\RoutingMiddleware->performRouting(Object(GuzzleHttp\Psr7\ServerRequest))
#1 /var/www/docker-aio/php/vendor/slim/csrf/src/Guard.php(476): Slim\Routing\RouteRunner->handle(Object(GuzzleHttp\Psr7\ServerRequest))
#2 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(168): Slim\Csrf\Guard->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Slim\Routing\RouteRunner))
#3 /var/www/docker-aio/php/vendor/slim/twig-view/src/TwigMiddleware.php(115): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
#4 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(121): Slim\Views\TwigMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
#5 /var/www/docker-aio/php/src/Middleware/AuthMiddleware.php(38): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
#6 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(269): AIO\Middleware\AuthMiddleware->__invoke(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
#7 /var/www/docker-aio/php/vendor/slim/slim/Slim/Middleware/ErrorMiddleware.php(76): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
#8 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(121): Slim\Middleware\ErrorMiddleware->process(Object(GuzzleHttp\Psr7\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
#9 /var/www/docker-aio/php/vendor/slim/slim/Slim/MiddlewareDispatcher.php(65): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(GuzzleHttp\Psr7\ServerRequest))
#10 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(199): Slim\MiddlewareDispatcher->handle(Object(GuzzleHttp\Psr7\ServerRequest))
#11 /var/www/docker-aio/php/vendor/slim/slim/Slim/App.php(183): Slim\App->handle(Object(GuzzleHttp\Psr7\ServerRequest))
#12 /var/www/docker-aio/php/public/index.php(181): Slim\App->run()
#13 {main}

Thanks,
Kai

[jtr]

Hi @beppi:

What type of hardware/infrastructure are you running your stack on (CPU, storage)?

What’s your CPU usage during these transactions? (check top/htop)

What do your actual Nextcloud logs contain during these events (not the mastercontainer)?

As for the memory, no, you don’t have to do anything. Trust the Linux kernel’s memory management. And don’t stress about a low “free” number - if you’re not familiar with Linux/modern OS memory management I can understand why that would appear stressful. In reality that’s just telling you how much more memory is available to allocate to… whatever it wants. If you have 7 GB under “available” than your buff/cache is likely 6.8 GB or something like that. You’ve got enough memory (at least in that moment in time snapshot you pulled those figures from).

In general, you should try to stop the various NC containers from the AIO interface.

When you say RAM starts ramping up, what metric are you using?

Oh, you didn’t post your config or app list and since it could turn out to be relevant: did you install any apps in NC or is it a default install still?

[beppi]

Hey thanks for the reply!

Updated post with server specs

Default AIO containers minus Talk, and another one I didn’t use.
Using the previewgenerator app as I thought it might help the problem of being unable to see any images unless I download and use a 3rd party image viewer

Idle htop: Mem ~2.1G/15.5G, no proc above 0.8% CPU
Throughout viewing of images: Very hard to say, but if I try to view an image, I most of the cores jump to around 15% usage which seems fine. However the image will never load. Uploading files never seems to go above 15% either. Same with uploading from mobile.

The memory ramping thing may still be an issue, but it doesnt continually increase anymore, just sits at a higher level after uploading.

I noticed that opening a folder with 8GB of photos, and scrolling causes all cores to jump to around 70% consistently during scrolling (only for inital loading of the entry), and then memory stays at about 5GB without decreasing (at least not immediately). From here, the page will freeze upon refresh or changing to another page within nextcloud, processes dont seem too strange in htop while this happens, and RAM stays around 5GB. Then I will get a timeout. After this, the RAM usage will increase slowly over time, around 0.02GB a minute. I cant access Nextcloud, unless I stop the apache container, log into the AIO interface, close all containers and restart them. Upon doing this, RAM usage drops down to 1GB

I’m sorry, but I’m unsure of how to give youo a config, but here is my docker-compose entry:

nextcloud:
image: nextcloud/all-in-one:latest
container_name: nextcloud-aio-mastercontainer
volumes:
- nextcloud_aio_mastercontainer:/mnt/docker-aio-config
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- APACHE_PORT=11000
- APACHE_IP_BINDING=0.0.0.0
- SKIP_DOMAIN_VALIDATION=true
- NEXTCLOUD_MEMORY_LIMIT=1024M
restart: unless-stopped
ports:
- 8080:8080
depends_on:
- app

[szaimen]

Hi, can you for a test disable imaginary from the AIO interface and check if that helps?

[beppi]

Wow, it seemed to at least significantly improve it RAM wise. Loading up the 8 GB folder of images still requires a lot of CPU power, but the ram jumps to 1.5 GB, then drops to 1.3 GB after all the files have been shown.

Worth noting though, there is another 8 GB folder, which the thumbnails seem to work/are already there, and opening this causes no noticeable change in CPU or RAM when scrolling.

It doesn’t seem to affect the thumbnail generation, but I can at least click on, and view images now, without having to restart the containers for even attempting to look at them!

Thanks so much! But what was it about Imaginary that was causing this?

[szaimen]

@beppi

But what was it about Imaginary that was causing this?

Yes, apparently. We have a fix scheduled for the next release which hopefully improves the situation.

[beppi]

You’re amazing, thank you for your hard work!

[beppi]

I will need to make a new issue for this, but while you’re still here: This doesn’t seem to fix the thumbnail generation, on web and on mobile, but at least on web I can click on the image to see it. On mobile however, I dont get a thumbnail, and I cant see it when clicking on it. It now instantly pops up a dialog saying “no resized image available”, and downloading the full image works, and shows it on screen. Is this related to imaginary at all? Do you know why this might be?