The Basics
- Nextcloud Server version:
- Nextcloud Hub 26 Winter(33.0.6) – AIO
- AIO-version:
- Nextcloud AIO v13.3.1
- Operating system and version:
- Ubuntu 24.04.4 LTS
Summary of the issue you are facing:
When enabling the LocalAI community container, I can access the webbackend, install downloaded models, but these remain super slow, w/o GPU-support. Any idea, how to enable GPU support? I tried to restart the container multiple times with various configurations + flags enabled (stop → rm → run) w/o success.
Steps to replicate it (hint: details matter!):
- stop containers via AIO interface
- enable LocaAI container
- Restart → w/o success…
I suspect, that GPU-access is somehow not passed to the local-ai container.
Following information in addition:
Command to start nextcloud-aio
~ ❯❯❯ sudo docker run --init --gpus all --sig-proxy=false --name nextcloud-aio-mastercontainer --restart always --publish 80:80 --publish 8080:8080 --publish 8443:8443 --volume nextcloud_aio_mastercontainer:/mnt/docker-aio-config --volume /var/run/docker.sock:/var/run/docker.sock:ro --env APACHE_IP_BINDING=0.0.0.0 --env APACHE_PORT=443 --env NEXTCLOUD_DATADIR="/mnt/data/nextcloud_data" --env NEXTCLOUD_ADDITIONAL_APKS="ocrmypdf tesseract-ocr-data-eng tesseract-ocr-data-deu tesseract-ocr-data-fra" --env NEXTCLOUD_ENABLE_NVIDIA_GPU=true nextcloud/all-in-one:latest
Details on GPU
~ ❯❯❯ nvidia-smi
Tue Jul 21 18:38:27 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4060 Ti Off | 00000000:01:00.0 Off | N/A |
| 0% 50C P8 10W / 165W | 154MiB / 16380MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 5371 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 5546 C+G ...c/gnome-remote-desktop-daemon 122MiB |
+-----------------------------------------------------------------------------------------+
Details on GPU 2.0
~ ❯❯❯ sudo docker run --rm --gpus all nvidia/cuda:12.9.0-base-ubuntu24.04 nvidia-smi ✘ 130
Tue Jul 21 16:39:26 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4060 Ti Off | 00000000:01:00.0 Off | N/A |
| 0% 50C P8 10W / 165W | 154MiB / 16380MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
Missing GPU for nextcloud-aio-local-ai
~ ❯❯❯ sudo docker inspect nextcloud-aio-local-ai --format '{{json .HostConfig.DeviceRequests}}'
null
Any hints how to resolve this would be appreciated.