Nextcloud on Docker on Virtual Machine High System Load

🏷️ General
, ,
1

I had Nextcloud 25.0.3 running in a Docker Container on a Lenovo ThinkCenter:

CPU: Intel(R) Core(TM) i3-4130T CPU @ 2.90GHz (4 cores)
Memory: 16 GB
Soild-state 500 GB Harddive

My Nextcloud setup has Maria DB 10.6.12 running in its own container for the database, and a letsencrypt container to proxy traffic.

All of this is connected via a Docker Compose File.

I recently acquired an HP ProLiant DL360 G7:

CPU x2: Intel(R) Xeon(R) CPU X5650 @ 2.67GHz (6 cores / 2 threads)
Memory:  72 GB
RAID 5 with 1.2 TB Storage

I split out a portion of HP ProLiant Server for a NixOS VM to be used as the new host for my Nextcloud Docker Containers using libvirt/KVM/virtmanager.

Virtual Machine specs:

CPU: 8 cores
Memory: 16 GB
Storage: 45 GB for OS and passthrough to host's 1.2 TB storage for persistent Docker volumes

Here is where I’m getting stuck:

I only have two users of my Nextcloud instance.

On the Lenovo ThinkCenter, the Nextcloud UI pages loads were under 1 second, and the “Load” under Administration → System was only between 0.15 - 0.80. Additionally, I only had local Memcache/APCu set up. Using docker stats the CPU for the Nextcloud web container, and all other Nextcloud-related Docker containers was below 1% with the occasional spike higher.

After moving Nextcloud to the Virtual Machine inside the HP ProLiant Server, pages take 7-10 seconds to load, and the “Load” under Administration → System is always above 2.0; usually between 2.5 - 5.8. The CPU usage via docker stats for the Nextcloud web container is always 8% and fluctuates between 16%-24% when idle. All other Nextcloud Docker Containers (letsencrypt, mariadb, redis, collabora) on the VM stay below 1% when idle. I have since added Redis for caching with no noticeable impact.

I have essentially moved my Nextcloud Docker containers from one machine to another with more resources, yet my performance is drastically worse :frowning:

Any ideas on how to troubleshoot this? Or anyone run into something similar? Why would only my Nextcloud Docker container be so drastically impacted by a move from one machine to another?

I’m stumped :frowning:

Any ideas? :bulb:

2

hi @AaronVonAwesome from your post I see you are not a rookie and trained enough to not ask dumb questions. I think you have to go the hard way turning every stone to see where the problem is located.

as educated guess I would check following things first

  • RAID/HDD - does it perform well? (a single SSD is much faster than RAID5 of spinning disks. hardware RAID controller have it’s own specialties e.g. missing write cache when battery is empty/missing dramatically slowing down write operations - which are enforced every second by mySQL)
  • did you maybe brake some caching during migration e.g. database index
  • are you missing some drivers?

increase the log level and check what the system is doing (maybe you need to analyze multiple parts one after another: application, web server, database)

3

@wwe - Thank you very much for the suggestions!

I will look into those and turn up the log level to see what is shows :+1:

I will also double check the raid controller; I was wondering about.

Question on the database suggestions - if the database indexes or database in general was faulty, would I not expect to see the database docker container show higher than normal load? But, I my case it’s strictly the Nextcloud web docker container.

Some other things I have tried:

  • ran occ db:add-missing-indices
  • ran occ files:scan
  • ran occ files:cleanup
  • ran occ files:scan-app-data

None of those found anything out of the ordinary :frowning:

Thank you again for the educated guesses; I’ll post updates as I find them.

4

Ok, I solved the issue, here is how:

Thanks to @wwe 's suggestion, I looked at the RAID 5 setup I have on the server.

The RAID controller checks out fine, and so what I ended up doing was splitting out the storage.

Here is what I mean by that.

I have a solid-state PCI harddive on my server, and that is actually where my Virtual Machine is housed. What that means is that the “harddrive” on the Virtual Machine is solid-state.

I adjusted by docker-compose file volumes so that only the Nextcould DATA folder was storing files to the RAID 5 array, but everything else, all the other volumes, where storing “locally” to the Virtual Machine’s storage. So the Nexcloud Config, MariaDB, etc. are now on essentially a solid-state device.

Thanks to Docker/Docker Compose this was extremely easy, and it solved my performance issues.

Here is my docker-compose.yaml file; hopefully that make it a bit clearer :innocent:

---
version: "2.1"

networks:

  nextcloud_web-net:
    external: false
    ipam:
      config:
        - subnet: 192.168.55.0/24
  nextcloud_db-net:
    external: false
  redis-net:
    external: false

services:

  letsencrypt:
    image: "linuxserver/swag:amd64-latest"
    hostname: "vonawesome"
    cap_add:
      - NET_ADMIN
    environment:
      PUID: "1001"
      PGID: "1001"
      TZ: "Asia/Singapore"
      URL: "vonawesome.cloud"
      SUBDOMAINS: "nc,nco,"
      VALIDATION: dns
      DNSPLUGIN: [pick your dns plugin] # see documentation: https://hub.docker.com/r/linuxserver/swag
      EMAIL: 'email@address.com'
      ONLY_SUBDOMAINS: true
      STAGING: false  # optional
      MAXMINDDB_LICENSE_KEY: ""
    ports:
      - "443:443"
    networks:
      nextcloud_web-net:
        ipv4_address: 192.168.55.5
    container_name: "swag-nextcloud"
    restart: unless-stopped
    volumes:
      - ./swag-nc-config/:/config/

  nextcloudweb:
    image: "linuxserver/nextcloud:amd64-latest"
    hostname: "vonawesome"
    environment:
      PUID: "1001"
      PGID: "1001"
      TZ: "Asia/Singapore"
      REDIS_HOST: "redis"
    expose:
      - "443"
      - "25"
    networks:
      nextcloud_web-net:
        ipv4_address: 192.168.55.6
      nextcloud_db-net:
      redis-net:
    extra_hosts:
      - "nco.vonawesom.cloud:192.168.55.5"
    container_name: "nextcloud-web"
    restart: unless-stopped
    depends_on:
      - "redis"
      - "letsencrypt"
      - "db"
      - "collabora"
    volumes:
      - ./nc-web-config/:/config/
      - ../nextcloud-storage/nc-web-data/:/data/  # This is actually a folder on a RAID 5 storage array

  redis:
    image: "redis:7-bullseye"
    environment:
      PUID: "1001"
      PGID: "1001"
      TZ: "Asia/Singapore"
    hostname: "vonawesome"
    container_name: "nextcloud-redis"
    networks:
      - redis-net
    expose:
      - "6379"
    restart: unless-stopped

  db:
    image: "linuxserver/mariadb:amd64-10.6.12"
    hostname: "vonawesome"
    environment:
      PUID: "1001"
      PGID: "1001"
      TZ: "Asia/Singapore"
      MYSQL_ROOT_PASSWORD: 'Keep it secret, keep it safe!'
      MYSQL_DATABASE: "vonawesomenextcloud"
      MYSQL_USER: "gandalf"
      MYSQL_PASSWORD: 'Keep it secret, keep it safe!'
    expose:
      - "3306"  # Default MariaDB port: 3306
    networks:
      nextcloud_db-net:
    container_name: "nextcloud-db"
    restart: unless-stopped
    volumes:
      - ./nc-db-mysql/:/config/

  collabora:
    image: "collabora/code:latest-amd64"
    environment:
      PUID: "1001"
      PGID: "1001"
      TZ: "Asia/Singapore"
      aliasgroup1: "https://nc\\.vonawesome\\.cloud:443"
      server_name: "nco.vonawesome.cloud"
    cap_add:
      - MKNOD
    privileged: true
    expose:
      - "443"
      - "9980"
    networks:
      nextcloud_web-net:
        ipv4_address: 192.168.55.8
    extra_hosts:
      - "nc.vonawesom.cloud:192.168.55.5"
    container_name: "nextcloud-office"
    restart: unless-stopped