Suspiciously Low Memory Usage

I have Nextcloud running in a Debian Proxmox lxc (basically this tutorial). Its taken a few weeks to get this far and I thought I had everything running well so I start to upload 1TB+ of photos. But my local upload speeds are super slow (<3Mbps). Windows desktop client says its gonna take days :melting_face:


My PC is on wifi so I’m not expecting LAN speeds, although they are slower than expected as well (<30Mbps). In either case, a 10X difference between what Iperf outputs seems like something is wrong.

I’ve added what I could from the Server Tuning documentation - APCu Caching + Redis File Locking (see Server config). And I even moved Cache Directory location to SSD rather than HDD pool.

Server configuration detail

CPU: 4 x Intel(R) Core™ i5-4690K CPU @ 3.50GHz (1 Socket)

Operating system: Linux 6.2.16-8-pve #1 SMP PREEMPT_DYNAMIC PMX 6.2.16-8 (2023-08-02T12:17Z) x86_64

Webserver: Apache 2.4.25 (apache2handler)

Database: mysql 10.5.19

PHP version: 8.1.22

Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, json, Reflection, SPL, session, standard, sodium, apache2handler, mysqlnd, PDO, xml, apcu, bcmath, calendar, ctype, curl, dom, mbstring, FFI, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imagick, imap, intl, ldap, exif, mysqli, pdo_mysql, Phar, posix, readline, redis, shmop, SimpleXML, sockets, sysvmsg, sysvsem, sysvshm, tokenizer, xmlreader, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 27.0.2 - 27.0.2.1

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: unknown

Signing status

List of activated apps
Enabled:
 - activity: 2.19.0
 - bruteforcesettings: 2.7.0
 - circles: 27.0.1
 - cloud_federation_api: 1.10.0
 - comments: 1.17.0
 - contactsinteraction: 1.8.0
 - dashboard: 7.7.0
 - dav: 1.27.0
 - federatedfilesharing: 1.17.0
 - federation: 1.17.0
 - files: 1.22.0
 - files_pdfviewer: 2.8.0
 - files_rightclick: 1.6.0
 - files_sharing: 1.19.0
 - files_trashbin: 1.17.0
 - files_versions: 1.20.0
 - firstrunwizard: 2.16.0
 - logreader: 2.12.0
 - lookup_server_connector: 1.15.0
 - memories: 5.4.0
 - nextcloud_announcements: 1.16.0
 - notifications: 2.15.0
 - oauth2: 1.15.1
 - password_policy: 1.17.0
 - photos: 2.3.0
 - previewgenerator: 5.3.0
 - privacy: 1.11.0
 - provisioning_api: 1.17.0
 - recognize: 4.3.2
 - recommendations: 1.6.0
 - related_resources: 1.2.0
 - serverinfo: 1.17.0
 - settings: 1.9.0
 - sharebymail: 1.17.0
 - support: 1.10.0
 - survey_client: 1.15.0
 - systemtags: 1.17.0
 - text: 3.8.0
 - theming: 2.2.0
 - twofactor_backupcodes: 1.16.0
 - updatenotification: 1.17.0
 - user_status: 1.7.0
 - viewer: 2.1.0
 - weather_status: 1.7.0
 - workflowengine: 2.9.0
Disabled:
 - admin_audit
 - encryption
 - facerecognition
 - files_external
 - suspicious_login
 - twofactor_totp
 - user_ldap

Configuration (config/config.php)
{
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "localhost",
        "cloud.REDACTED.com",
        "192.168.1.113"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "dbtype": "mysql",
    "version": "27.0.2.1",
    "overwrite.cli.url": "http:\/\/localhost",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "default_phone_region": "US",
    "mysql.utf8mb4": true,
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "memcache.local": "\\OC\\Memcache\\APCu",
    "redis": {
        "host": "***REMOVED SENSITIVE VALUE***",
        "port": 0,
        "timeout": 0
    },
    "filelocking.enabled": true,
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "updater.secret": "***REMOVED SENSITIVE VALUE***",
    "maintenance": false,
    "theme": "",
    "loglevel": 2,
    "memories.exiftool": "\/var\/www\/nextcloud\/apps\/memories\/exiftool-bin\/exiftool-amd64-glibc",
    "memories.vod.path": "\/var\/www\/nextcloud\/apps\/memories\/exiftool-bin\/go-vod-amd64",
    "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
    "memories.vod.ffprobe": "\/usr\/bin\/ffprobe",
    "enabledPreviewProviders": [
        "OC\\Preview\\Image",
        "OC\\Preview\\HEIC",
        "OC\\Preview\\TIFF",
        "OC\\Preview\\Movie"
    ],
    "preview_max_x": 2048,
    "preview_max_y": 2048,
    "memories.gis_type": 1,
    "cache_path": "\/cache\/nextcloud"
}

Cron Configuration: Array
(
[backgroundjobs_mode] => cron
[lastcron] => 1692636603
)

# crontab -u -www-data -l
*/5 * * * * php -f /var/www/nextcloud/cron.php --define apc.enable_cli=1
*/10 * * * * php /var/www/nextcloud/occ preview:pre-generate

External storages: files_external is disabled

Encryption: no

User-backends:

  • OC\User\Database

Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36

It seems like there is more than enough CPU, RAM, and bandwidth to spare. I just can’t figure out how to get Nextlcloud to use it. CPU usage hardly goes over 5% and suspiciously, Memory hovers around 256MB even though I have 16GB.
Which makes me think that’s the bottleneck and there’s some setting somewhere that’s limiting the memory.

I’ve set PHP memory limit to 2GB in both /etc/php/8.1/cli/php.ini and /etc/php/8.1/apache2/php.ini and thats what show up in Nextcloud console:

When I research memory related issues its mostly posts about Nextcloud using too much RAM :thinking:

Is this low level of memory usage normal?
Or am I off base thinking memory is the cause of my slow upload speeds?

Any advice or additional diagnosis recommendations would be great!
Thanks!

Nextcloud Logs: pastebin .com/ n9CPzNqk
Apache Error Logs: pastebin .com/ L8JZReG8

Is your connection - even though you’re local - through the public IP address (or a hostname that maps to your public IP address) by chance?

I don’t think so, I do have a cloudflare tunnel container set up, but I turned it off to rule out that case.
Screenshot 2023-08-24 194810
Just now I even turned off my modem to make sure, and still got the same results.

I am using a domain name, but my pihole resolves it locally.
Screenshot 2023-08-24 195058

The turnkey nextcloud appliance is build upon the turnkey lamp stack, that comes with the apache2handler module (libapache2-mod-php) and not with the much faster php-fpm (cgi-fcgi module)

Here some reasons, why fastcgi is advantageous:

  1. Isolation: PHP-FPM runs PHP scripts in separate worker processes, isolated from the web server.
  2. Fine-Grained Control: PHP-FPM offers fine-grained control over PHP processes.
  3. Efficient Resource Utilization: PHP-FPM allows for more efficient use of server resources. In contrast, the mod_php approach (using “apache2handler”) embeds PHP directly into the Apache process, which is less flexible and efficient in managing resources.
  4. Improved Performance: PHP-FPM can improve overall system performance, especially in high-traffic scenarios. It processes PHP scripts faster and can handle more simultaneous requests by distributing the workload across multiple PHP worker processes.
  5. Enhanced Security: Running PHP in separate processes enhances security.
  6. http2 support

Regarding the low RAM usage, I assume that the database is not optimally optimized on the available memory.

With a total of 16GB RAM, you can use the following values, for example:

innodb_buffer_pool_size = 12G
max_allowed_packet = 2G
key_buffer_size = 256M
max_connections = 200

But that’s just a glitch, you have to deal with it carefully and try out what optimizes and accelerates for you


I hope that helps,
much luck!

2 Likes