Internal Server Error after Ubuntu updates

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 20.0.5): 24.0.7
Operating system and version (eg, Ubuntu 20.04): Ubuntu 20.04
Apache or nginx version (eg, Apache 2.4.25): Apache 2.4.41-4ubuntu3.12
PHP version (eg, 7.4): 7.4.33

The issue you are facing:
Since updating my Ubuntu server (apt upgrade) I get:


The server encountered an internal error and was unable to complete your request.
Please contact the server administrator if this error reappears multiple times, please include the technical details below in your report.
More details can be found in the server log.

Is this the first time you’ve seen this error? (Y/N): Y

Steps to replicate it:

  1. navigate to my nextcloud url

I am able to reach the Apache2 default page with the IP address, and can get php.info with IP address and /info.php.

No errors appear in the apache2 logs when refreshing either the IP address or the URL.

I get the entry:

166.62.236.50 - - [22/Nov/2022:19:14:02 +0000] "GET /index.php/apps/dashboard/ HTTP/1.1" 500 1072 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"

in the nextcloud.access file when I refresh the URL.

The output of your Nextcloud log in Admin > Logging:
/var/log/apache2/nextcloud.access

206.12.14.216 - - [22/Nov/2022:19:00:14 +0000] "GET /status.php HTTP/1.1" 500 5004 "-" "Mozilla/5.0 (Macintosh) mirall/3.4.3git (build 9390) (Nextcloud, osx-19.6.0 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
103.121.209.224 - - [22/Nov/2022:19:00:17 +0000] "HEAD / HTTP/1.1" 500 4982 "http://www.uptimedoctor.com/monitored-by-user-kdyson" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36 www.uptimedoctor.com (username kdyson)"
166.62.236.50 - - [22/Nov/2022:19:00:44 +0000] "GET /index.php/apps/dashboard/ HTTP/1.1" 500 5514 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
166.62.236.50 - - [22/Nov/2022:19:00:51 +0000] "GET /index.php/apps/dashboard/ HTTP/1.1" 500 1072 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
166.62.236.50 - - [22/Nov/2022:19:00:54 +0000] "GET /index.php/apps/dashboard/ HTTP/1.1" 500 1072 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
206.12.14.216 - - [22/Nov/2022:19:01:16 +0000] "GET /status.php HTTP/1.1" 500 5004 "-" "Mozilla/5.0 (Macintosh) mirall/3.4.3git (build 9390) (Nextcloud, osx-19.6.0 ClientArchitecture: x86_64 OsArchitecture: x86_64)"

tail -f /var/log/apache2/nextcloud.error

[Tue Nov 22 17:37:21.256138 2022] [proxy_fcgi:error] [pid 16020] [client 142.103.118.1:57184] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught OCP\\HintException: [0]: Memcache \\OC\\Memcache\\Redis not available for local cache (Is the matching PHP module installed and enabled?)\n\n  thrown in /var/www/nextcloud/lib/private/Memcache/Factory.php on line 90'
[Tue Nov 22 17:37:27.283163 2022] [proxy_fcgi:error] [pid 16021] [client 142.103.118.1:22886] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught OCP\\HintException: [0]: Memcache \\OC\\Memcache\\Redis not available for local cache (Is the matching PHP module installed and enabled?)\n\n  thrown in /var/www/nextcloud/lib/private/Memcache/Factory.php on line 90'
[Tue Nov 22 17:37:33.313682 2022] [proxy_fcgi:error] [pid 16031] [client 142.103.118.1:57209] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught OCP\\HintException: [0]: Memcache \\OC\\Memcache\\Redis not available for local cache (Is the matching PHP module installed and enabled?)\n\n  thrown in /var/www/nextcloud/lib/private/Memcache/Factory.php on line 90'
[Tue Nov 22 17:37:39.338569 2022] [proxy_fcgi:error] [pid 16023] [client 142.103.118.1:26873] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught OCP\\HintException: [0]: Memcache \\OC\\Memcache\\Redis not available for local cache (Is the matching PHP module installed and enabled?)\n\n  thrown in /var/www/nextcloud/lib/private/Memcache/Factory.php on line 90'
[Tue Nov 22 17:37:50.834689 2022] [proxy_fcgi:error] [pid 16022] [client 142.103.118.1:11993] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught OCP\\HintException: [0]: Memcache \\OC\\Memcache\\Redis not available for local cache (Is the matching PHP module installed and enabled?)\n\n  thrown in /var/www/nextcloud/lib/private/Memcache/Factory.php on line 90'
[Tue Nov 22 17:37:57.200361 2022] [proxy_fcgi:error] [pid 16080] [client 142.103.118.1:63553] AH01071: Got error 'PHP message: PHP Fatal error:  Uncaught OCP\\HintException: [0]: Memcache \\OC\\Memcache\\Redis not available for local cache (Is the matching PHP module installed and enabled?)\n\n  thrown in /var/www/nextcloud/lib/private/Memcache/Factory.php on line 90'
[Tue Nov 22 18:02:22.278538 2022] [proxy_fcgi:error] [pid 928] [client 208.115.199.21:42162] AH01071: Got error 'Primary script unknown', referer: https://hub.cnsp-rcps.ca/staus.php
[Tue Nov 22 18:22:54.228643 2022] [proxy_fcgi:error] [pid 929] [client 208.115.199.21:52204] AH01071: Got error 'Primary script unknown', referer: https://hub.cnsp-rcps.ca/staus.php
[Tue Nov 22 18:25:33.392492 2022] [proxy_fcgi:error] [pid 927] [client 120.76.135.15:34712] AH01071: Got error 'Primary script unknown', referer: http://hub.cnsp-rcps.ca/wp-login.php
[Tue Nov 22 18:55:10.203325 2022] [proxy_fcgi:error] [pid 927] [client 208.115.199.21:36418] AH01071: Got error 'Primary script unknown', referer: https://hub.cnsp-rcps.ca/staus.php

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'instance',
  'passwordsalt' => '5+password',
  'secret' => 'secret/c0b3Uk8Zlf',
  'trusted_domains' => 
  array (
    0 => 'hub.cnsp-rcps.ca',
  ),
  'datadirectory' => '/home/ubuntu/data',
  'dbtype' => 'mysql',
  'version' => '24.0.7.1',
  'overwrite.cli.url' => 'https://hub.cnsp-rcps.ca',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'admin',
  'dbpassword' => 'password',
  'installed' => true,
  'mail_from_address' => 'admin.cnsp.hub',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => 'use.startmail.com',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,
  'mail_smtpsecure' => 'ssl',
  'mail_smtphost' => 'smtp.mail.com',
  'mail_smtpport' => '465',
  'mail_smtpname' => 'me@mail.com',
  'mail_smtppassword' => 'password',
  'default_phone_region' => 'ISO 3166-2:CA',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => array (
    'host' => 'localhost',
    'port' => 6379,
  ),
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
 'updater.release.channel' => 'stable',
  'updater.secret' => '$2y$10$NVccI0LP4PY.826VUOuJaua9ct8xj1rKSagO7uvofIm2zr81MDpNG',
);

The output of your Apache/nginx/system log in /var/log/____:
tail -f /var/log/apache2/access.log

143.244.50.176 - - [22/Nov/2022:16:35:19 +0000] "POST /boaform/admin/formLogin HTTP/1.1" 404 492 "http://206.12.94.186:80/admin/login.asp" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:71.0) Gecko/20100101 Firefox/71.0"
166.62.236.50 - - [22/Nov/2022:18:00:45 +0000] "GET /info.php HTTP/1.1" 200 23077 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
141.255.166.2 - - [22/Nov/2022:18:16:50 +0000] "GET / HTTP/1.1" 200 11173 "-" "Hello World"
128.14.209.162 - - [22/Nov/2022:18:43:06 +0000] "GET / HTTP/1.1" 200 3421 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36"
152.89.196.103 - - [22/Nov/2022:18:45:08 +0000] "\x16\x03\x01" 400 497 "-" "-"
166.62.236.50 - - [22/Nov/2022:18:54:31 +0000] "GET /info.php HTTP/1.1" 200 23080 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
166.62.236.50 - - [22/Nov/2022:18:54:34 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
166.62.236.50 - - [22/Nov/2022:18:57:36 +0000] "GET /info.php HTTP/1.1" 200 23078 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
166.62.236.50 - - [22/Nov/2022:18:57:40 +0000] "GET / HTTP/1.1" 200 3476 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"
166.62.236.50 - - [22/Nov/2022:19:02:45 +0000] "GET /info.php HTTP/1.1" 200 23080 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0"

tail -f /var/log/apache2/error.log

[Tue Nov 22 14:43:41.807438 2022] [core:notice] [pid 15934] AH00094: Command line: '/usr/sbin/apache2'
[Tue Nov 22 14:44:02.272876 2022] [mpm_prefork:notice] [pid 15934] AH00171: Graceful restart requested, doing restart
[Tue Nov 22 14:44:02.283531 2022] [mpm_prefork:notice] [pid 15934] AH00163: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f configured -- resuming normal operations
[Tue Nov 22 14:44:02.283544 2022] [core:notice] [pid 15934] AH00094: Command line: '/usr/sbin/apache2'
[Tue Nov 22 14:44:02.594228 2022] [mpm_prefork:notice] [pid 15934] AH00171: Graceful restart requested, doing restart
[Tue Nov 22 14:44:02.604165 2022] [mpm_prefork:notice] [pid 15934] AH00163: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f configured -- resuming normal operations
[Tue Nov 22 14:44:02.604183 2022] [core:notice] [pid 15934] AH00094: Command line: '/usr/sbin/apache2'
[Tue Nov 22 17:38:22.273225 2022] [mpm_prefork:notice] [pid 15934] AH00169: caught SIGTERM, shutting down
[Tue Nov 22 17:59:46.996254 2022] [mpm_prefork:notice] [pid 925] AH00163: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f configured -- resuming normal operations
[Tue Nov 22 17:59:47.013526 2022] [core:notice] [pid 925] AH00094: Command line: '/usr/sbin/apache2'

Other evidence:
Using the command apache2 gives me this message:

[Tue Nov 22 19:17:39.206534 2022] [core:warn] [pid 3163] AH00111: Config variable ${APACHE_RUN_DIR} is not defined
apache2: Syntax error on line 80 of /etc/apache2/apache2.conf: DefaultRuntimeDir must be a valid directory, absolute or relative to ServerRoot

I see error messages related to Redis, but I have it installed to this version:

Redis server v=5.0.7 sha=00000000:0 malloc=jemalloc-5.2.1 bits=64 build=66bd629f924ac924

When I navigate to https://hub.cnsp-rcps.ca/staus.php, as indicated in the error:

'Primary script unknown', referer: https://hub.cnsp-rcps.ca/staus.php

I get a page that simply says “File not found.”

I have an image of the working instance from about a year ago, but I am hoping to get this working once again before attempting the nuclear option and rolling back to that image.

Thanks in advance for any help.

Hi,

Config variable ${APACHE_RUN_DIR} is not defined

Your problem is that you haven’t set a proper hostname inside apache2.conf
https://stackoverflow.com/questions/72105163/apache-http-server-config-variable-is-not-defined-error-how-to-solve

DefaultRuntimeDir must be a valid directory

Du you have /var/run/apache2 directory? If not create it.

‘Primary script unknown’, referer: https://hub.cnsp-rcps.ca/staus.php

Shouldn’t it be …staTus.php ?