Exceed 30 DB connections regularly

ℹ️ Support
,
1

Steps to reproduce

Browse to directory of many images that do not yet have thumbnails (suspect one session per image thumbnail that requires generation)

Use files app (suspect generating previews involves one session per file)

Use Bookmarks app (suspect one session per folder, app often complains of being unable to count bookmarks for short periods)

Given my suspicions, and I am quite happy to be shown to be wrong, the javascript in the client side needs to restrain itself if there is possibility of having many requests outstanding OR NextCloud needs some form of DB Connection Pooling, if it doesn’t already have such.

Expected behaviour

Not to exceed 30 DB Connections.
Sheesh, I would hate to see what happens if hundreds of people where using my instance of NextCloud if it can’t even cope with one (me).

Actual behaviour

Requests fail as DB Connections are refused.

Files app - previews unavailable

Photos app - no thumbnail shown

Bookmarks app - temporary failure to count bookmarks

Server configuration detail

Operating system: Linux 2.6.32-954.3.5.lve1.4.78.el6.x86_64 #1 SMP Thu Mar 26 08:20:27 EDT 2020 x86_64

Webserver: LiteSpeed (litespeed)

Database: mysql 5.6.50

PHP version:

7.4.13
Modules loaded: Core, date, libxml, openssl, pcre, sqlite3, zlib, bz2, calendar, ctype, curl, hash, filter, ftp, gettext, gmp, SPL, iconv, pcntl, readline, Reflection, session, standard, shmop, SimpleXML, mbstring, tokenizer, xml, litespeed, bcmath, dom, fileinfo, gd, imagick, imap, intl, json, exif, mysqli, mysqlnd, PDO, pdo_mysql, pdo_sqlite, Phar, posix, soap, sockets, timezonedb, xmlreader, xmlrpc, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 20.0.3 - 20.0.3.2

Updated from an older Nextcloud/ownCloud or fresh install: 20.0.2

Where did you install Nextcloud from: Local Gentoo Linux then uploaded to shared hosting after local testing

Signing status

Array
(
)

List of activated apps 
Enabled:
 - accessibility: 1.6.0
 - activity: 2.13.4
 - admin_audit: 1.10.0
 - apporder: 0.11.0
 - bookmarks: 4.0.5
 - calendar: 2.1.2
 - cloud_federation_api: 1.3.0
 - comments: 1.10.0
 - contacts: 3.4.2
 - contactsinteraction: 1.1.0
 - dashboard: 7.0.0
 - dav: 1.16.1
 - extract: 1.3.0
 - federatedfilesharing: 1.10.1
 - files: 1.15.0
 - files_markdown: 2.3.1
 - files_pdfviewer: 2.0.1
 - files_rightclick: 0.17.0
 - files_sharing: 1.12.0
 - files_texteditor: 2.14.0
 - files_videoplayer: 1.9.0
 - firstrunwizard: 2.9.0
 - issuetemplate: 0.7.0
 - keeweb: 0.6.4
 - logreader: 2.5.0
 - lookup_server_connector: 1.8.0
 - notes: 4.0.2
 - notifications: 2.8.0
 - oauth2: 1.8.0
 - password_policy: 1.10.1
 - photos: 1.2.1
 - privacy: 1.4.0
 - provisioning_api: 1.10.0
 - recommendations: 0.8.0
 - serverinfo: 1.10.0
 - settings: 1.2.0
 - sharebymail: 1.10.0
 - support: 1.3.0
 - survey_client: 1.8.0
 - systemtags: 1.10.0
 - tasks: 0.13.6
 - text: 3.1.0
 - theming: 1.11.0
 - twofactor_backupcodes: 1.9.0
 - updatenotification: 1.10.0
 - user_status: 1.0.1
 - viewer: 1.4.0
 - weather_status: 1.0.0
 - workflowengine: 2.2.0
Disabled:
 - camerarawpreviews
 - encryption
 - federation
 - files_external
 - files_trashbin
 - files_versions
 - nextcloud_announcements
 - user_ldap
Configuration (config/config.php) 
{
    "installed": true,
    "dbtype": "mysql",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbtableprefix": "oc_",
    "forcessl": false,
    "theme": "",
    "3rdpartyroot": "",
    "3rdpartyurl": "",
    "blacklisted_files": [
        ".htaccess"
    ],
    "default_language": "en",
    "default_locale": "en_AU",
    "overwritehost": "",
    "overwriteprotocol": "https",
    "overwritewebroot": "",
    "overwritecondaddr": "",
    "overwrite.cli.url": "https:\/\/cloud.remotely-helpful.com\/",
    "htaccess.RewriteBase": "\/",
    "defaultapp": "dashboard",
    "knowledgebaseenabled": true,
    "knowledgebaseurl": "http:\/\/api.apps.nextcloud.com\/v1",
    "appstoreenabled": true,
    "appstoreurl": "https:\/\/apps.nextcloud.com\/api\/v1",
    "apps_paths": [
        {
            "path": "\/home\/remotely\/www\/cloud\/apps",
            "url": "\/apps",
            "writable": true
        }
    ],
    "mail_smtpdebug": false,
    "mail_smtpmode": "smtp",
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpport": "587",
    "mail_smtptimeout": 10,
    "mail_smtpauthtype": "LOGIN",
    "appcodechecker": true,
    "log_type": "nextcloud",
    "logfile": "\/home\/remotely\/cloud\/logs\/nextcloud.log",
    "loglevel": 2,
    "logdateformat": "F d, Y H:i:s",
    "logtimezone": "Australia\/Sydney",
    "log_rotate_size": 104857600,
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "hashingCost": 10,
    "trashbin_retention_obligation": "180, auto",
    "allow_user_to_change_display_name": true,
    "xframe_restriction": true,
    "custom_csp_policy": "default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src *; font-src 'self' data:; media-src *",
    "customclient_desktop": "",
    "customclient_android": "",
    "customclient_ios": "",
    "remember_login_cookie_lifetime": 1296000,
    "session_keepalive": true,
    "updatechecker": true,
    "updater.server.url": "https:\/\/updates.nextcloud.com\/updater_server\/",
    "updater.release.channel": "stable",
    "has_internet_connection": true,
    "writable_appsdir": true,
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "version": "20.0.3.2",
    "maxZipInputSize": "838860800",
    "allowZipDownload": true,
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "maintenance": false,
    "trusted_domains": [
        "cloud.remotely-helpful.info",
        "cloud.remotely-helpful.com",
        "remotely-helpful.info",
        "remotely-helpful.com",
        "remotelyhelpful.info",
        "remotelyhelpful.com",
        "remotelyhelpful.info.au",
        "remotelyhelpful.com.au",
        "remotely-helpful.info.au",
        "remotely-helpful.com.au",
        "vmres08.auserver.com.au"
    ],
    "secret": "***REMOVED SENSITIVE VALUE***",
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "appstore.experimental.enabled": true,
    "mail_smtpauth": 1,
    "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
    "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
    "app_install_overwrite": [
        "keeweb",
        "apporder",
        "calendar",
        "contacts",
        "external",
        "camerarawpreviews",
        "tasks",
        "bookmarks",
        "occweb",
        "extract",
        "gpxedit"
    ],
    "mysql.utf8mb4": false,
    "enabledPreviewProviders": [
        "OC\\Preview\\PNG",
        "OC\\Preview\\JPEG",
        "OC\\Preview\\GIF",
        "OC\\Preview\\HEIC",
        "OC\\Preview\\BMP",
        "OC\\Preview\\XBitmap",
        "OC\\Preview\\MP3",
        "OC\\Preview\\TXT",
        "OC\\Preview\\MarkDown",
        "OC\\Preview\\OpenDocument",
        "OC\\Preview\\Krita"
    ],
    "tempdirectory": "\/home\/remotely\/tmp\/NextCloud"
}

Are you using external storage, if yes which one: local/smb/sftp/…

Are you using encryption:

Are you using an external user-backend, if yes which one:

Client configuration

Browser: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36

Operating system: Gentoo Linux

Logs

Web server error log 
203.122.235.20 - lyallp [06/Jan/2021:14:05:11 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 248 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:11 +1100] "GET /ocs/v2.php/core/navigation/apps?absolute=true&format=json HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:14 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 237 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:14 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 1085 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:15 +1100] "PROPFIND /remote.php/dav/files/lyallp/KeePass HTTP/1.1" 207 1240 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:15 +1100] "GET /remote.php/dav/files/lyallp/KeePass/LyallPearce.kdbx HTTP/1.1" 200 872270 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:16 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 237 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:43 +1100] "GET /ocs/v2.php/core/navigation/apps?absolute=true&format=json HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:05:43 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 248 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:14:29:14 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:14:29:14 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 13976 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:14:29:23 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp HTTP/2" 207 261 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:14:29:25 +1100] "PROPFIND /remote.php/dav/principals/users/lyallp/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:14:29:25 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/ HTTP/2" 207 366 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:14:29:25 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 337 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:14:29:25 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 6283 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:15:29:15 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:15:29:15 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 13976 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:15:29:20 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp HTTP/2" 207 261 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:15:29:24 +1100] "PROPFIND /remote.php/dav/principals/users/lyallp/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:15:29:25 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/ HTTP/2" 207 366 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:15:29:26 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 337 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:15:29:26 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 6283 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:16:06:09 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 237 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:16:06:10 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 1080 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:16:06:10 +1100] "PROPFIND /remote.php/dav/files/lyallp/KeePass HTTP/1.1" 207 1237 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:16:06:11 +1100] "GET /remote.php/dav/files/lyallp/KeePass/LyallPearce.kdbx HTTP/1.1" 200 872494 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:16:06:12 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 237 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:16:06:42 +1100] "PROPFIND /remote.php/dav/files/lyallp/ HTTP/1.1" 207 250 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:16:06:42 +1100] "GET /ocs/v2.php/core/navigation/apps?absolute=true&format=json HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh) mirall/3.0.3git (build 3799) (Nextcloud)"
203.122.235.20 - lyallp [06/Jan/2021:16:29:15 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:16:29:15 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 13976 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:16:29:26 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp HTTP/2" 207 261 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:16:29:26 +1100] "PROPFIND /remote.php/dav/principals/users/lyallp/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:16:29:26 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/ HTTP/2" 207 366 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:16:29:26 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 337 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:16:29:26 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 6283 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:16 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:16 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 14025 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:18 +1100] "REPORT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 1723 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:19 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp HTTP/2" 207 261 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:19 +1100] "PROPFIND /remote.php/dav/principals/users/lyallp/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:19 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/ HTTP/2" 207 366 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:19 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 337 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:29:19 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 6308 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:23 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/ HTTP/2" 207 14025 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/D6CCB494-85B2-465F-A9BD-4CACEF15FC82.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/C6D5E655-EA30-0001-6FCB-44E618D11932.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/c3608e37-5977-4f3d-b272-8e643182695d.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/83aac79a-76a6-42fb-9c25-d3c84be7e86e.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/d8db2d38e7.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/2093AFE3-2AB8-4FB2-A3F9-509426F623C4.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/3EF94B43-B652-4D7B-B7EB-378403E41780.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/66f6c02d-425f-4bf7-b485-f79722a86461.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/B0F79DCC-4FD5-44E6-8284-B98397E23A32.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:24 +1100] "PUT /remote.php/dav/addressbooks/users/lyallp/lyall_pearce/101FCF98-FA0A-40B2-948E-2682F2394FED.vcf HTTP/2" 204 0 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:27 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp HTTP/2" 207 261 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:27 +1100] "PROPFIND /remote.php/dav/principals/users/lyallp/ HTTP/2" 207 285 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:27 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/ HTTP/2" 207 366 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
203.122.235.20 - lyallp [06/Jan/2021:17:35:27 +1100] "PROPFIND /remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/ HTTP/2" 207 337 "https://cloud.remotely-helpful.info/remote.php/dav/addressbooks/users/lyallp/z-app-generated--contactsinteraction--recent/" "Thunderbird CardBook/55.0"
Nextcloud log 
{"reqId":"mFN47yvqP394WvR7icg3","level":3,"time":"2021-01-06T15:37:16+11:00","remoteAddr":"203.122.235.20","user":"--","app":"core","method":"GET","url":"/apps/bookmarks/folder/33/count","message":{"Exception":"Doctrine\\DBAL\\DBALException","Message":"Failed to connect to the database: An exception occurred in driver: SQLSTATE[42000] [1226] User 'remotely_owncld' has exceeded the 'max_user_connections' resource (current value: 30)","Code":0,"Trace":[{"file":"/home/remotely/public_html/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":1449,"function":"connect","class":"OC\\DB\\Connection","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php","line":892,"function":"getWrappedConnection","class":"Doctrine\\DBAL\\Connection","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/lib/private/DB/Connection.php","line":202,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->","args":["SELECT * FROM `oc_appconfig`",[],[],null]},{"file":"/home/remotely/public_html/cloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php","line":206,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->","args":["SELECT * FROM `oc_appconfig`",[],[]]},{"file":"/home/remotely/public_html/cloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":217,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/lib/private/AppConfig.php","line":345,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/lib/private/AppConfig.php","line":110,"function":"loadConfigValues","class":"OC\\AppConfig","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/lib/private/AppConfig.php","line":301,"function":"getApps","class":"OC\\AppConfig","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/lib/private/legacy/OC_App.php","line":949,"function":"getValues","class":"OC\\AppConfig","type":"->","args":[false,"installed_version"]},{"file":"/home/remotely/public_html/cloud/lib/private/TemplateLayout.php","line":185,"function":"getAppVersions","class":"OC_App","type":"::","args":[]},{"file":"/home/remotely/public_html/cloud/lib/private/legacy/OC_Template.php","line":183,"function":"__construct","class":"OC\\TemplateLayout","type":"->","args":["error",""]},{"file":"/home/remotely/public_html/cloud/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/lib/private/legacy/OC_Template.php","line":332,"function":"printPage","class":"OC\\Template\\Base","type":"->","args":[]},{"file":"/home/remotely/public_html/cloud/index.php","line":63,"function":"printExceptionErrorPage","class":"OC_Template","type":"::","args":[{"__class__":"Doctrine\\DBAL\\DBALException"},500]}],"File":"/home/remotely/public_html/cloud/lib/private/DB/Connection.php","Line":72,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0","version":"20.0.3.2","id":"5ff582ca1674a"}
Browser log
2

Well, raise the limit. Problem is that for many actions, you need the database, there needs to be an update for new generated thumbnails, if you want to show certain data etc.

Is there a huge improvement in terms of performance? It’s probably not easy to realize especially when you except the interface to be responsive. And you have all the apps where you haven’t much control.

3

Easy to say ‘well, raise the limit.’ Not a problem on my test Gentoo Linux, at home.

However, I am on shared hosting and I can’t even get a re-build of ImageMagick with HEIC support nor MySQL multi-byte changes to support NextCloud unicode, let alone increasing my DB connections - I may adversely affect other users of the shared hosting environment.

I understand the position of my shared hosting provider, however painful it is but am unwilling to cough up the $ for a private server in which I could ‘raise the limit’. (nearly 3 times more expensive than shared hosting for this provider).

I am also quite happy with my hosting provider, given the above limitations.

I feel, and I am not a contributor to NextCloud, so I try not to sound like a petulant child, that demands NextCloud should be configurable in terms of maximum number of concurrent DB Connections it will request, and not assume the universe is available and at it’s disposal.

Do the apps use an API to NextCloud? Surely there is an API for DB access and some form of pooling can be dropped between it, with a ‘wait for Connection to become available’ or existing behaviour of ‘exceeded DB Connection limit’ with immediate return.

I appreciate the time you have taken to respond to my issue. I feel that this is a NextCloud scalability issue. You can tell I am old school can’t you? :slightly_smiling_face:

4

On most hosted platforms, it’s more or less painful to run Nextcloud depending on their configuration.

This is effectively one point you could get something changed. For that reason I asked about the performance improvements for the database pooling approach. If there is an important gain, it may be worth to think about it also if there are a lot of redundant and unnecessary db queries, …

A few times, there were questions about the performance of the raspberry pi, which in general is not the best but it is great to see the impact of different changes because you can easily stress it and others can easily reproduce the environment. That is hard for hosted environment where a lot of things are hidden and can’t be changed.

5

I saw that a few of the DB connection failures where sql queries to obtain the oc_appconfig, no where clause, so all rows, in my case over 300 rows. Excuse my ignorance but could the oc_appconfig table be ‘cached’ somewhere rather than requiring DB queries each time?

A dumb question but generate some PHP which contains the value of these rows, which is loaded at request startup, rather than connecting to a DB and executing the query every time? An API call could then re-generate this PHP on demand, by the administrator?

Maybe DB connection pooling could cache the oc_appconfig table, performing a simple check on the incoming query to see if it’s oc_appconfig query and returning a pre-loaded result (with a cache clear method so the pre-loaded result can be cleared and reloaded).

After all, a simple string compare will use a lot less processing power than parsing, optimising and executing SQL queries.

Forgive me if these are silly questions, I have no idea as to the internal workings of NextCloud and grateful to those that do.

6

Yes, you can configure caching in your database. That improves the speed quite a lot.

On top of that, there is caching php level. You haven’t set it up yet, not sure if that is possible in your environment: Memory caching — Nextcloud latest Administration Manual latest documentation