Updated to 25.0.1.1 from 24. Error message: Service Unavailable

ℹ️ Support
1
Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Reverse Proxy Server
Nextcloud version: 25.0.0.1
Operating system and version : Ubuntu 22.04.1 LTS
Apache: Apache/2.4.52 (Ubuntu)
PHP version: PHP 8.1.13(cli)

Apache Backend Sever
Operating system and version : Ubuntu 22.04.1 LTS
Apache: Apache/2.4.52 (Ubuntu)
PHP version: PHP 8.0.25(cli)

The issue you are facing: Update Nextcloud from 24 to 25.

Error message: Service Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Is this the first time you’ve seen this error? : YES, since nextcloud 10 never had this error.

Steps to replicate it:

  1. Browes to: nextcloud.allenintech.com

The output of your Nextcloud log in Admin > Logging:

I have complete access to servers

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => 'xxxxxxxxxxxxx',
  'passwordsalt' => 'xxxxxxxxxxx,
  'secret' => 'xxxxxxxxxxxxxxxx,
  'trusted_domains' => 
  array (
    0 => 'nextcloud.allenintech.com',
    1 => 'www.nextcloud.allenintech.com',
  ),
  'trusted_proxies' => 
  array (
    0 => '127.0.0.1',
    1 => '10.0.0.1',
  ),
  'datadirectory' => '/var/www/html/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '25.0.1.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'xxxxxx',
  'dbpassword' => 'xxxxxx',
  'logtimezone' => 'UTC',
  'installed' => true,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'memcached_servers' => 
  array (
    0 => 
    array (
      0 => 'localhost',
      1 => 11211,
    ),
    1 => 
    array (
      0 => 'nextcloud.allenintech.com',
      1 => 11211,
    ),
  ),
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
  'overwriteprotocol' => 'https',
  'overwrite.cli.url' => 'https://nextcloud.allenintech.com',
  'overwritehost' => 'nextcloud.allenintech.com',
  'default_phone_region' => 'US',
  'mail_smtpmode' => 'sendmail',
  'mail_from_address' => 'eatobiasa',
  'mail_domain' => 'gmail.com',
  'mail_smtphost' => 'smtp.gmail.com',
  'mail_smtpport' => '465',
  'mail_smtptimeout' => 60,
  'updater.release.channel' => 'stable',
  'mail_sendmailmode' => 'smtp',
  'mysql.utf8mb4' => true,
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'eatobiasa@gmail.com',
  'mail_smtppassword' => 'xxxxxxxxxx',
  'mail_smtpsecure' => 'ssl',
  'mail_smtpauthtype' => 'PLAIN',
  'app_install_overwrite' => 
  array (
    0 => 'drawio',
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

Reverse Proxy syslog
root@genesis:/var/log# tail -n 10 syslog
Nov 30 18:47:36 genesis kernel: [60457.714927] audit: type=1300 audit(1669852056.736:1901): arch=c000003e syscall=1 success=yes exit=4 a0=3 a1=7ffd21301100 a2=4 a3=0 items=0 ppid=1023 pid=54151 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=259 comm="sshd" exe="/usr/sbin/sshd" subj=unconfined key=(null)
Nov 30 18:47:36 genesis kernel: [60457.714930] audit: type=1327 audit(1669852056.736:1901): proctitle=737368643A206772616365205B707269765D
Nov 30 18:47:36 genesis systemd[1]: Started Session 259 of User grace.
Nov 30 18:47:37 genesis kernel: [60458.127077] audit: type=1105 audit(1669852057.148:1902): pid=54151 uid=0 auid=1000 ses=259 subj=unconfined msg='op=PAM:session_open grantors=pam_selinux,pam_loginuid,pam_keyinit,pam_permit,pam_umask,pam_unix,pam_systemd,pam_mail,pam_limits,pam_env,pam_env,pam_selinux acct="grace" exe="/usr/sbin/sshd" hostname=10.0.0.42 addr=10.0.0.42 terminal=ssh res=success'
Nov 30 18:47:37 genesis kernel: [60458.129887] audit: type=1103 audit(1669852057.152:1903): pid=54252 uid=0 auid=1000 ses=259 subj=unconfined msg='op=PAM:setcred grantors=pam_permit,pam_cap acct="grace" exe="/usr/sbin/sshd" hostname=10.0.0.42 addr=10.0.0.42 terminal=ssh res=success'
Nov 30 18:47:37 genesis kernel: [60458.136783] audit: type=1112 audit(1669852057.160:1904): pid=54151 uid=0 auid=1000 ses=259 subj=unconfined msg='op=login id=1000 exe="/usr/sbin/sshd" hostname=10.0.0.42 addr=10.0.0.42 terminal=/dev/pts/4 res=success'
Nov 30 18:47:46 genesis kernel: [60467.140159] audit: type=1100 audit(1669852066.160:1905): pid=54262 uid=1000 auid=1000 ses=259 subj=unconfined msg='op=PAM:authentication grantors=pam_permit,pam_cap acct="root" exe="/usr/bin/su" hostname=? addr=? terminal=/dev/pts/4 res=success'
Nov 30 18:47:46 genesis kernel: [60467.140206] audit: type=1101 audit(1669852066.160:1906): pid=54262 uid=1000 auid=1000 ses=259 subj=unconfined msg='op=PAM:accounting grantors=pam_permit acct="root" exe="/usr/bin/su" hostname=? addr=? terminal=/dev/pts/4 res=success'
Nov 30 18:47:46 genesis kernel: [60467.140401] audit: type=1103 audit(1669852066.160:1907): pid=54262 uid=1000 auid=1000 ses=259 subj=unconfined msg='op=PAM:setcred grantors=pam_permit,pam_cap acct="root" exe="/usr/bin/su" hostname=? addr=? terminal=/dev/pts/4 res=success'
Nov 30 18:47:46 genesis kernel: [60467.142289] audit: type=1105 audit(1669852066.164:1908): pid=54262 uid=1000 auid=1000 ses=259 subj=unconfined msg='op=PAM:session_open grantors=pam_env,pam_env,pam_mail,pam_limits,pam_permit,pam_umask,pam_unix,pam_systemd acct="root" exe="/usr/bin/su" hostname=? addr=? terminal=/dev/pts/4 res=success'

Reverse Proxy error-nextcloud.log

tail -n 10 error-nextcloud.log
[Wed Nov 30 12:11:18.495936 2022] [proxy:error] [pid 50702] (111)Connection refused: AH00957: http: attempt to connect to 10.0.0.42:80 (10.0.0.42) failed
[Wed Nov 30 12:11:18.495968 2022] [proxy_http:error] [pid 50702] [client 10.0.0.45:62378] AH01114: HTTP: failed to make connection to backend: 10.0.0.42
[Wed Nov 30 12:11:19.944639 2022] [proxy:error] [pid 50704] (111)Connection refused: AH00957: http: attempt to connect to 10.0.0.42:80 (10.0.0.42) failed
[Wed Nov 30 12:11:19.944669 2022] [proxy_http:error] [pid 50704] [client 10.0.0.45:62379] AH01114: HTTP: failed to make connection to backend: 10.0.0.42
[Wed Nov 30 12:11:20.598509 2022] [proxy:error] [pid 50703] (111)Connection refused: AH00957: http: attempt to connect to 10.0.0.42:80 (10.0.0.42) failed
[Wed Nov 30 12:11:20.598537 2022] [proxy_http:error] [pid 50703] [client 10.0.0.45:62380] AH01114: HTTP: failed to make connection to backend: 10.0.0.42
[Wed Nov 30 12:11:20.921490 2022] [proxy:error] [pid 50700] (111)Connection refused: AH00957: http: attempt to connect to 10.0.0.42:80 (10.0.0.42) failed
[Wed Nov 30 12:11:20.921518 2022] [proxy_http:error] [pid 50700] [client 10.0.0.45:62381] AH01114: HTTP: failed to make connection to backend: 10.0.0.42
[Wed Nov 30 12:11:21.338157 2022] [proxy:error] [pid 50701] (111)Connection refused: AH00957: http: attempt to connect to 10.0.0.42:80 (10.0.0.42) failed
[Wed Nov 30 12:11:21.338185 2022] [proxy_http:error] [pid 50701] [client 10.0.0.45:62382] AH01114: HTTP: failed to make connection to backend: 10.0.0.42

Reverse Proxy Server

<VirtualHost *:80>
ServerName “www.nextcloud.allenintech.com”
ServerAlias “nextcloud.allenintech.com”
UseCanonicalName On
Redirect permanent / https://nextcloud.allenintech.com/
#RewriteEngine on
#RewriteCond %{SERVER_NAME} =www.nextcloud.allenintech.com [OR]
#RewriteCond %{SERVER_NAME} =nextcloud.allenintech.com
#RewriteRule ^https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

#Secure nextcloud

<VirtualHost *:443>
ServerName “www.nextcloud.allenintech.com”
ServerAlias “nextcloud.allenintech.com”
LogLevel warn
LogFormat “%v %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"” vhost_combined
CustomLog “/var/log/apache2/access-nextcloud.log” vhost_combined
ErrorLog “/var/log/apache2/error-nextcloud.log”
UseCanonicalName on
ProxyRequests Off
ProxyPreserveHost On
ProxyPass / http://10.0.0.42/
#ProxyPassReverse / http://nextcloud.allenintech.com/
ProxyPassReverse / http://10.0.0.42/
SSLEngine On
SSLProxyEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder on
# Compression is disabled by default on my distribution (CentOS 6)
# SSLCompression off
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS

#RewriteEngine on
#RewriteRule ^/\.well-known/carddav https://www.nextcloud.allenintech.com/remote.php/dav/ [R=301,L]
#RewriteRule ^/\.well-known/caldav  https://%{SERVER_NAME}/remote.php/dav/ [R=301,L]

<IfModule mod_headers.c>
	Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
</IfModule>

Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/nextcloud.allenintech.com-0001/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/nextcloud.allenintech.com-0001/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/nextcloud.allenintech.com-0001/chain.pem
```

Backend Server

<VirtualHost *:443>
ServerAdmin admin@shekinah
ServerName www.nextcloud.allenintech.com
ServerAlias nextcloud.allenintech.com
UseCanonicalName on
DocumentRoot “/var/www/html/nextcloud”
LogLevel warn
LogFormat “%v %h %l %u %t "%r" %>s %b "%{Referer}i" "%User-Agent}i"” vhost_combined
ErrorLog “/var/log/apache2/error-nextcloud-proxy.log”
CustomLog “/var/log/apache2/access-nextcloud-proxy.log” vhost_common

<Directory /var/www/html/nextcloud>
	Require all granted
#	Options +FollowSymlinks
	Options FollowSymlinks Multiviews
	AllowOverride ALL 
	#Order allow,deny
	#allow from all 
	<IfModule mod_dav.c>
		Dav off
	</IfModule>
</Directory>
#<IfModule mod_rewrite.c>
	LogLevel alert rewrite:trace3
	#-------------------------------------------------------------
	RewriteEngine on
	RewriteRule ^\.well-known/carddav http://%{SERVER_NAME}/remote.php/dav/ [R=301,L]
	RewriteRule ^\.well-known/caldav  http://%{SERVER_NAME}/remote.php/dav/ [R=301,L]
	#-------------------------------------------------------------
#</IfModule>
#<IfModule mod_rewrite.c>
	#-------------------------------------------------------------
	RewriteEngine on
	RewriteRule ^\.well-known/carddav   /var/www/html/nextcloud/remote.php/dav/ [R=301,L]
	RewriteRule ^\.well-known/caldav    /var/www/html/nextcloud/remote.php/dav/ [R=301,L]
	RewriteRule ^\.well-known/webfinger /var/www/html/nextcloud/index.php/.well-known/webfinger [R=301,L]
	RewriteRule ^\.well-known/nodeinfo  /var/www/html/nextcloud/index.php/.well-known/nodeinfo [R=301,L]
	#-------------------------------------------------------------
#</IfModule>

RewriteRule ^\.well-known/carddav /var/www/html/nextcloud/remote.php/dev/ [R=301,L]
RewriteRule ^\.well-known/caldav  /var/www/html/nextcloud/remote.php/dev/ [R=301,L]
2

The solution’s bottom line is:
**1) Install apc for the php module **

  • (a) apt install php8.1-apcu

2) update php.ini with apc.enable_cli = 1

  • Make sure to update the correct php.ini! The correct php.ini is under cli under the php version which you are running enabled. In this case: /etc/php/8.1/cli/php.ini.

  • (b) add apc.enable_cli = 1 to the bottom of php.ini.

         **Caveats To Getting To The Solution**

Do not trust update-alternatives --config php. It will appear to set the php version and php -v will indicate that version but phpinfo will be running a different version!

To set php configurations and modules use
a2en[mod|conf] or a2dis[mod|conf] to enable or disable php modules

Except for the php version to be used, disable all other php modules and configurations. For example do not have enabled php8.0, php8.1, php8.0-fpm, php8.1-fpm, php8.0-apcu and php8.1-apcu . Decide the php version and disable all other php version modules and corresponding configurations. In this case enable php8.1, php8.1-fpm, php8.1-apcu.

If nextcloud was working prior to upgrading Ubuntu then most likely the errors messages Service Unavailable and Internal Server Error are the result of the aforementioned, mixed php versions enabled and apcu not being installed. Also occ will not work. So changing firewall or nextcloud’s …config/config.php settings are probably not necessary.

Finally, Service Unavailable and Internal Server Error will not render nextcloud. However, to check the php version running a simple web page can be rendered from nextcloud’s root directory to display phpinfo()

  1. Make a simple web page to display phpinfo to determine the running php version