Internal Server Error after upgrading to NextCloud 22.2.3

Nextcloud version (eg, 20.0.5): 22.2.3
Operating system and version (eg, Ubuntu 20.04): Unsure, as this is a shared hosting;
Apache or nginx version (eg, Apache 2.4.25): Apache 2.4.51
PHP version (eg, 7.4): 7.4.x

The issue you are facing:
After successively upgrading from NextCloud 19.x to 22.2.3, alternating between point-release upgrades and full versions, after the last upgrade, trying to load the NextCloud homepage yields an error 500:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable >to complete your request.

Please contact the server administrator at webmaster@domain.tld to inform them of the time this error occurred, and the actions you performed just before this error.
More information about this error may be available in the server error log.
Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. Upgrade from NextCloud 19 to latest 19.x point release
  2. Upgrade from 19.x to 20 [Repeat, until you reach the current 22.2.3 version]
  3. Upgrade all apps from outside NextCloud, as should be done after a core upgrade
  4. Try to open homepage

The output of your Nextcloud log in Admin > Logging:

´Not available as GUI inaccessible.

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'Instance_id',
  'passwordsalt' => 'salt',
  'secret' => 'another_long_secret',
  'trusted_domains' => 
  array (
    0 => 'nc.domain.tld',
  ),
  'datadirectory' => '/home/domain/nc_data',
  'dbtype' => 'mysql',
  'version' => '22.2.3.0',
  'overwrite.cli.url' => 'https://nc.domain.tld',
  'dbname' => 'dbname_nc',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'dbuser_nc',
  'dbpassword' => 'db_password',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 0,
  'twofactor_enforced' => 'true',
  'twofactor_enforced_groups' => 
  array (
    0 => 'tfa',
  ),
  'twofactor_enforced_excluded_groups' => 
  array (
    0 => 'admin',
  ),
  'app_install_overwrite' => 
  array (
    0 => 'radio',
    1 => 'files_external_gdrive',
    2 => 'files_external_onedrive',
    3 => 'files_ebookreader',
    4 => 'dropit',
    5 => 'files_frommail',
    6 => 'ownbackup',
    7 => 'ocr',
  ),
  'updater.secret' => 'this_is_a_long_secret_string',
  'has_rebuilt_cache' => true,
  'encryption.legacy_format_support' => true,
  'encryption.key_storage_migrated' => false,
);

The output of your Apache/nginx/system log in /var/log/____:

Not sure, on a shared hosting these files are usually unavailable. However, ther’s a “Latest web server error log messages” accessible through cPanel

[Thu Nov 25 10:12:19.561858 2021] [access_compat:error] [pid 118042:tid 47026625881856] [client 38.23.240.68:50358] AH01797: client denied by server configuration: /home/domain/public_html/nc.domain.tld/config
[Thu Nov 25 10:11:47.581807 2021] [access_compat:error] [pid 117752:tid 47026627983104] [client 38.23.240.68:50302] AH01797: client denied by server configuration: /home/domain/public_html/nc.domain.tld/config
[Thu Nov 25 10:10:54.605172 2021] [access_compat:error] [pid 117752:tid 47026627983104] [client 38.23.240.68:50284] AH01797: client denied by server configuration: /home/domain/public_html/nc.domain.tld/config

Please uninstall the app dropit

sudo -u www-data php occ app:remove dropit

https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/occ_command.html#apps-commands-label

Just done that. Still the same error 500. Another way to tell what’s going wrong?

Uninstall all not supported app.

Isn’t the upgrade process supposed to automatically disable unsupported apps?
And, how do I get a list of unsupported apps? The manual doesn’t say, and the logs I have access don’t tell anything either. A few are disabled already:

  • audioplayer
  • audioplayer_editor
  • bookmarks
  • calendar
  • checksum
  • drop_account
  • event_update_notification
  • external
  • files_accesscontrol
  • files_ebookreader
  • files_external_dropbox
  • files_external_gdrive
  • files_frommail
  • files_fulltextsearch
  • files_fulltextsearch_tesseract
  • fulltextsearch
  • groupfolders
  • keeporsweep
  • maps
  • ocr
  • ocsms
  • ownbackup
  • pdfdraw
  • phonetrack
  • polls
  • previewgenerator
  • printer
  • quota_warning
  • radio
  • ransomware_detection
  • ransomware_protection
  • registration
  • spreed
  • support
  • user_ldap
  • weather
  • workflow_pdf_converter

Some are still enabled. Anyway, from a virtual terminal within cPanel, I did:
./occ app:update --all
No result, still Error 500. I disabled a few, but the names of those remaining appear important enough that I didn’t touch them:
Enabled:

  • admin_audit: 1.12.0
  • bruteforcesettings: 2.2.0
  • cloud_federation_api: 1.5.0
  • dashboard: 7.2.0
  • dav: 1.19.0
  • encryption: 2.10.0
  • federatedfilesharing: 1.12.0
  • files: 1.17.0
  • files_linkeditor: 1.1.6
  • files_markdown: 2.3.4
  • files_mindmap: 0.0.25
  • files_pdfviewer: 2.3.1
  • files_readmemd: 1.2.1
  • files_rightclick: 1.1.0
  • files_sharing: 1.14.0
  • files_trashbin: 1.12.0
  • files_versions: 1.15.0
  • files_videoplayer: 1.11.0
  • firstrunwizard: 2.11.0
  • lookup_server_connector: 1.10.0
  • nextcloud_announcements: 1.11.0
  • oauth2: 1.10.0
  • password_policy: 1.12.0
  • passwords: 2021.11.20
  • privacy: 1.6.0
  • provisioning_api: 1.12.0
  • serverinfo: 1.12.0
  • settings: 1.4.0
  • suspicious_login: 4.0.0
  • twofactor_backupcodes: 1.11.0
  • updatenotification: 1.12.0
  • user_external: 2.1.0
  • user_status: 1.2.0
  • viewer: 1.6.0
  • workflowengine: 2.4.0

Followed by:
./occ maintenance:mode --on
./occ maintenance:repair
./occ maintenance:mode --off

Still no success :frowning: Interestingly, even when I turn on maintenance mode from the terminal, the same error 500 is displayed instead of the normal splash screen.

Well, since it appears next to impossible to repair a failed NextCloud installation (as frequently happens during these constant upgraded pushed out before being fully tested), I solved the issue the crude way: by restoring one 3 backup snapshots taken at the same time: data folder, executable folder, database.

Worked well in my case since there wasn’t much activity on this instance.

I then proceeded on upgrading step by step from the web GUI, after disabling all applications, very slowly, upgrading then disabling apps each time, until reaching 21.x. Given the issues with the upgrade process, I still consider 22.x to be beta-quality. Only then did I activate the apps.

In a few words: make sure your hosting or yourself have a solid backup strategy even before installing NextCloud.