Nextcloud server refuses to start

ℹ️ Support
, , , , ,
1
Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version : 26.0.1.1
Operating system and version (eg, Ubuntu 20.04):

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Apache or nginx version (eg, Apache 2.4.25): 2.4.56-1~deb11u2
PHP version:

PHP 8.1.18 (cli) (built: Apr 14 2023 04:39:46) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.18, Copyright (c) Zend Technologies
    with Zend OPcache v8.1.18, Copyright (c), by Zend Technologies

The issue you are facing:

Apche2 is not starting.
Found using journalctl -xe


░░ A start job for unit apache2.service has begun execution.
░░ 
░░ The job identifier is 1103.
Dec 27 14:51:23  petra apachectl[6943]: AH00526: Syntax error on line 12 of /etc/apache2/sites-enabled/nextcloud.conf:
Dec 27 14:51:23 petra apachectl[6943]: SSLCertificateFile: file '/etc/letsencrypt/live/nc.XXX.duckdns.org/fullchain.pem' does not exist or is empty
Dec 27 14:51:23 petra apachectl[6940]: Action 'start' failed.
Dec 27 14:51:23 petra apachectl[6940]: The Apache error log may have more information.
Dec 27 14:51:23 petra systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStart= process belonging to unit apache2.service has exited.
░░

It loos like there is a .../DOMAINNAME/fullcheain.pem file missing
Is this the first time you’ve seen this error? (Y/N):

Steps to replicate it:

  1. restart apache2

The output of your Nextcloud log in Admin > Logging:

Well, NC-server is not running, is there another way to show this

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'passwordsalt' => 'xyzzy',
  'secret' => 'xyzzy,
  'trusted_domains' => 
  array (
    0 => 'localhost',
    11 => '217.120.233.176',
    1 => '192.168.2.67',
    14 => 'petra',
    3 => 'petra',
  ),
  'datadirectory' => '/opt/ncdata/data',
  'dbtype' => 'mysql',
  'version' => '26.0.1.1',
  'overwrite.cli.url' => 'https://petra/',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'ncadmin',
  'dbpassword' => 'YPAJv2tM1zLvrmcOnSk4r4H4kFW7/oGLp3Ouc/iI8B0=',
  'installed' => true,
  'instanceid' => 'ocosndegi5bo',
  'tempdirectory' => '/opt/ncdata/data/tmp',
  'preview_max_x' => '2048',
  'preview_max_y' => '2048',
  'jpeg_quality' => '60',
  'overwriteprotocol' => 'https',
  'maintenance' => false,
  'logfile' => '/opt/ncdata/data/nextcloud.log',
  'trusted_proxies' => 
  array (
    11 => '127.0.0.1',
    12 => '::1',
    13 => 'petra',
    14 => '192.168.2.67',
  ),
  'default_phone_region' => 'NL',
  'loglevel' => '2',
  'log_type' => 'file',
  'htaccess.RewriteBase' => '/',
  'mail_sendmailmode' => 'smtp',
  'mail_smtpdebug' => true,
  'mail_from_address' => 'nextcloud',
  'mail_smtpmode' => 'smtp',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpport' => '587',
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'xxxx@xxxxxxx.nl',
  'mail_smtppassword' => 'xyzzy',
  'mail_smtpstreamoptions' => 
  array (
    'ssl' => 
    array (
      'allow_self_signed' => true,
      'verify_peer' => false,
      'verify_peer_name' => false,
    ),
  ),
  'filelocking.enabled' => true,
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
    'timeout' => 0.0,
    'password' =>  'xyzzy',
  ),
  'mail_smtpsecure' => 'tls',
  'theme' => '',
  'data-fingerprint' => 'xyzzy',
);

The output of your Apache/nginx/system log in /var/log/____:
There is no apache.log , yet I past the /var/log/apache2/error.log

[Mon Oct 30 20:45:04.251897 2023] [proxy_fcgi:error] [pid 195294:tid 548089590144] [remote 192.168.2.40:36402] AH01071: Got error 'PHP message: PHP Warning:  Array to st
ring conversion in /var/www/ncp-web/index.php on line 276'
[Mon Oct 30 20:45:12.176113 2023] [proxy_fcgi:error] [pid 195294:tid 548224094592] [remote 192.168.2.40:36402] AH01071: Got error 'PHP message: PHP Warning:  Array to st
ring conversion in /var/www/ncp-web/index.php on line 276'
[Mon Oct 30 20:54:02.573313 2023] [proxy_fcgi:error] [pid 195294:tid 548140044672] [remote 192.168.2.40:36402] AH01071: Got error 'PHP message: PHP Warning:  Array to st
ring conversion in /var/www/ncp-web/index.php on line 276'
[Tue Nov 14 12:45:06.483802 2023] [proxy_fcgi:error] [pid 195295:tid 548123226496] [remote 192.168.2.40:49540] AH01071: Got error 'PHP message: PHP Warning:  Array to st
ring conversion in /var/www/ncp-web/index.php on line 276'
[Tue Nov 14 14:56:49.433975 2023] [http2:warn] [pid 195295:tid 547465716096] [client 192.168.2.40:49540] h2_stream(195295-2396-31,CLEANUP): started=1, scheduled=1, ready
=0, out_buffer=0
[Tue Nov 14 14:57:49.469389 2023] [http2:warn] [pid 195295:tid 547465716096] [client 192.168.2.40:49540] h2_stream(195295-2396-31,CLEANUP): started=1, scheduled=1, ready
=0, out_buffer=0
[Tue Nov 14 14:58:49.469578 2023] [http2:warn] [pid 195295:tid 547465716096] [client 192.168.2.40:49540] h2_stream(195295-2396-31,CLEANUP): started=1, scheduled=1, ready
=0, out_buffer=0
[Tue Nov 14 14:59:49.472453 2023] [http2:warn] [pid 195295:tid 547465716096] [client 192.168.2.40:49540] h2_stream(195295-2396-31,CLEANUP): started=1, scheduled=1, ready
=0, out_buffer=0

and /var/log/apache2/nc-error.log

[Mon Dec 11 06:02:07.741121 2023] [proxy_fcgi:error] [pid 195295:tid 547625079168] (70008)Partial results are valid but processing is incomplete: [client 121.64.82.54:59704] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 06:44:26.763546 2023] [proxy_fcgi:error] [pid 195294:tid 547935478144] (70008)Partial results are valid but processing is incomplete: [client 121.64.82.54:54364] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 06:44:26.801874 2023] [proxy_fcgi:error] [pid 195294:tid 547927085440] (70008)Partial results are valid but processing is incomplete: [client 121.64.82.54:54378] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 08:55:51.773350 2023] [proxy_fcgi:error] [pid 195294:tid 547566363008] (70008)Partial results are valid but processing is incomplete: [client 121.64.82.54:37754] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 09:37:32.436177 2023] [proxy_fcgi:error] [pid 195295:tid 547432145280] (70007)The timeout specified has expired: [client 121.64.82.54:39378] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 09:39:55.475484 2023] [proxy_fcgi:error] [pid 195295:tid 547423752576] (70007)The timeout specified has expired: [client 121.64.82.54:39358] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 10:07:56.433580 2023] [proxy_fcgi:error] [pid 195294:tid 547591541120] (70008)Partial results are valid but processing is incomplete: [client 121.64.82.54:58566] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 10:11:56.592116 2023] [proxy_fcgi:error] [pid 195295:tid 547432145280] (70008)Partial results are valid but processing is incomplete: [client 121.64.82.54:52674] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 11:38:20.938068 2023] [proxy_fcgi:error] [pid 195295:tid 547658649984] (70007)The timeout specified has expired: [client 109.37.147.235:11842] AH01075: Error dispatching request to :61042: (reading input brigade)
[Mon Dec 11 12:36:41.829321 2023] [proxy_fcgi:error] [pid 195295:tid 547675435392] (70008)Partial results are valid but processing is incomplete: [client 121.64.82.54:38280] AH01075: Error dispatching request to :61042: (reading input brigade)

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.
2

This indicates that you have configured the website using a certificate from Let’s Encrypt but it is missing. Did you ever successfully run Certbot to obtain a certificate?

And this indicates that PHP took longer as expected to process the request. Apache tried to get a response from the PHP-FPM service via proxy_cgi, but did never get one.

Without any further details about that machine I would suggest to check your DNS configuration - maybe a missing or defct DNS resolver is causing a timeout in PHP-FPM which would also explain why Certbot did not create a TLS certificate yet. But his is just an educated guess. To really solve this, I would need to see the machine in detail.

Edit:

Due to the time stamps these errors seem not related to the other problems.

The first errors indicate, that that an ICS calendar was provided to be read but it does not follow the formal standard. Maybe the response was an error (due to the defect DNS setup).

The last error referring to Redis indicates that you configured to use Redis in Nextcloud but the Redis server is not running or not configured properly (using Unix Sockets and not an IP connection).

3

Just so you know. The logfiles contain a lot personal data or secrets which you propably don’t want to share.
It is better to remove them. They are not necessary for the solution.

4

Indeed since a couple of weeks I do have a different modem that for sure does not support some of the DNS functions as is required. That needs to be fixed.

I set up the system using Nextcloud PI. I assume running certbot is taken care of by NextcloudPI . So where is a place where I can find more information about running certbot.

5

Sorry, I don’t know anything about Nextcloud PI at all. Maybe the documentation at https://nextcloudpi.com/ can help you.

6

So, holiday season is over and therefor time to get this problem fixed.
I plowed through the log files and found this is in daemon.log

Jan  3 09:35:21 reina apachectl[603]: AH00526: Syntax error on line 12 of /etc/apache2/sites-enabled/nextcloud.conf:
Jan  3 09:35:21 reina apachectl[603]: SSLCertificateFile: file '/etc/letsencrypt/live/nc.XXX.duckdns.org/fullchain.pem' does not exist or is empty
Jan  3 09:35:21 reina apachectl[556]: Action 'start' failed.
Jan  3 09:35:21 reina apachectl[556]: The Apache error log may have more information.

There is no apache error.log , so no clue there and indeed /etc/letsencrypt/live/ is empty.

Are there any manual commands to get the certificate files in place ?

Thanks in advance, YaNn

7

Hi @yann1420, have you a certificate already generated for your domain?
if not, you can use Certbot to create certificates.

8

No, there is no certificate.
I understand this can be done using CertBot, but Is the process also described somewhere? Because I do not feel secure.
Thank you.

9

@yann1420 this is the link to instructions to download and use : Certbot Instructions | Certbot
You can choose to automatically redirect your site to HTTPS and the software will create the config based on the first config for 80 port (HTTP)

10

Running the procedure as per your suggestion and the following error messages:

$ sudo certbot --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Error while running apache2ctl configtest.
Action 'configtest' failed.
The Apache error log may have more information.

AH00526: Syntax error on line 12 of /etc/apache2/sites-enabled/nextcloud.conf:
SSLCertificateFile: file '/etc/letsencrypt/live/nc.X.duckdns.org/fullchain.pem' does not exist or is empty

The apache plugin is not working; there may be problems with your existing configuration.
The error was: MisconfigurationError("Error while running apache2ctl configtest.\nAction 'configtest' failed.\nThe Apache error log may have more information.\n\nAH00526: Syntax error on line 12 of /etc/apache2/sites-enabled/nextcloud.conf:\nSSLCertificateFile: file '/etc/letsencrypt/live/nc.titania.duckdns.org/fullchain.pem' does not exist or is empty\n")
$

Apparently there is something wrong with the Apache2 config file /etc/apache2/sites-enabled/nextcloud.conf . Therefor here is the content, what is wrong with it:

### DO NOT EDIT! THIS FILE HAS BEEN AUTOMATICALLY GENERATED. CHANGES WILL BE OVERWRITTEN ###

<IfModule mod_ssl.c>
  <VirtualHost _default_:443>
    DocumentRoot /var/www/nextcloud
    CustomLog /var/log/apache2/nc-access.log combined
    ErrorLog  /var/log/apache2/nc-error.log
    SSLEngine on
    SSLProxyEngine on
    SSLCertificateFile  	/etc/ssl/certs/ssl-cert-snakeoil.pem
    SSLCertificateKeyFile	/etc/ssl/private/ssl-cert-snakeoil.key
    SSLCertificateFile		/etc/letsencrypt/live/nc.X.duckdns.org/fullchain.pem
    SSLCertificateKeyFile	/etc/letsencrypt/live/nc.X.duckdns.org/privkey.pem

    # For notify_push app in NC21
    ProxyPass /push/ws ws://127.0.0.1:7867/ws
    ProxyPass /push/ http://127.0.0.1:7867/
    ProxyPassReverse /push/ http://127.0.0.1:7867/
  </VirtualHost>

  <Directory /var/www/nextcloud/>
    Options +FollowSymlinks
    AllowOverride All
    <IfModule mod_dav.c>
      Dav off
    </IfModule>
    LimitRequestBody 0
    SSLRenegBufferSize 10486000
  </Directory>
  <IfModule mod_headers.c>
    Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains"
  </IfModule>
</IfModule>

And indeed the file /etc/letsencrypt/live/nc.X.duckdns.org/fullchain.pem does not exist.

11

Hi @yann1420, to get only certificate you just have to run this: sudo certbot certonly --apache and then add the new certificate to your https config.

The issue is the same. when you use cerbot --apache and select to redirect HTTP to https( automatically) you have to remove https configuration and check your 80 (HTTP) config because it will create a new config for https and check it before creating certificates.

12

Well - of course you can not start Apache configured with HTTPS before having a certificate available. Usually the site is first set up without HTTPS and then Certbot will update the site configuration when it is used for the first time for that domain and if Apache integration is enabled.

13

Per you suggestion:

# sudo certbot -v  certonly --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Please enter the domain name(s) you would like on your certificate (comma and/or
space separated) (Enter 'c' to cancel): X.duckdns.org
Requesting a certificate for X.duckdns.org
Performing the following challenges:
http-01 challenge for X.duckdns.org
Waiting for verification...
Challenge failed for domain X.duckdns.org
http-01 challenge for X.duckdns.org

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: X.duckdns.org
  Type:   connection
  Detail: x.y.zz.y: Fetching http://X.duckdns.org/.well-known/acme-challenge/H1GZpo9ahseM-Z5a8rAgwCMBxTs9h04MSAQHO45d32Y: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

I included the the tail of the .../letsencrypt.log file:

2024-01-05 17:28:45,541:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/snap/certbot/3567/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/main.py", line 1869, in main
    return config.func(config, plugins)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/main.py", line 1600, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/main.py", line 143, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/client.py", line 517, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/snap/certbot/3567/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2024-01-05 17:28:45,544:ERROR:certbot._internal.log:Some challenges have failed.
14

That is what I figured, but I have no clue how a proper .../sites-available/nextcloud.conf looks like.

15

@yann1420 can you share your nextcloud.conf for 80 port ( HTTP) and hide personal informations

16 
<VirtualHost *:80>
  DocumentRoot /var/www/nextcloud/
  ServerName	202.44.44.202

  <Directory /var/www/nextcloud/>
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews

    <IfModule mod_dav.c>
      Dav off
    </IfModule>
  </Directory>
RewriteEngine on
RewriteCond %{SERVER_NAME} =202.44.44.202 [OR]
RewriteCond %{SERVER_NAME} =office.not365.net
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

Linux extcloud 6.1.0-16-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.67-1 (2023-12-12) x86_64 GNU/Linux

17

Hi,
Is X.duckdns.org reachable from the outside?

18

this is the compleet .../sites-available/nextcloud.conf

### DO NOT EDIT! THIS FILE HAS BEEN AUTOMATICALLY GENERATED. CHANGES WILL BE OVERWRITTEN ###

<IfModule mod_ssl.c>
  <VirtualHost _default_:443>
    DocumentRoot /var/www/nextcloud
    CustomLog /var/log/apache2/nc-access.log combined
    ErrorLog  /var/log/apache2/nc-error.log
    SSLEngine on
    SSLProxyEngine on
    SSLCertificateFile  	/etc/ssl/certs/ssl-cert-snakeoil.pem
    SSLCertificateKeyFile	/etc/ssl/private/ssl-cert-snakeoil.key
    SSLCertificateFile		/etc/letsencrypt/live/nc.X.duckdns.org/fullchain.pem
    SSLCertificateKeyFile	/etc/letsencrypt/live/nc.X.duckdns.org/privkey.pem

    # For notify_push app in NC21
    ProxyPass /push/ws ws://127.0.0.1:7867/ws
    ProxyPass /push/ http://127.0.0.1:7867/
    ProxyPassReverse /push/ http://127.0.0.1:7867/
  </VirtualHost>

  <Directory /var/www/nextcloud/>
    Options +FollowSymlinks
    AllowOverride All
    <IfModule mod_dav.c>
      Dav off
    </IfModule>
    LimitRequestBody 0
    SSLRenegBufferSize 10486000
  </Directory>
  <IfModule mod_headers.c>
    Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains"
  </IfModule>
</IfModule>
19

I have port forwards for port 80 and 443 to my RPI4.

When accessing

  • local-RPI4:80 there is no response ( bust that is expected as the nextcloud.conf is not correct.
  • local-RPI4:443 I get:
Apache2 Debian Default Page
It works!

This is the default welcome page used to test the correct operation of the Apache2 server after installation on Debian systems. If you can read this page, it means that the Apache HTTP server installed at this site is working properly. You should replace this file (located at /var/www/html/index.html) before continuing to operate your HTTP server.
  • local-RPI4:4443 I get:
Bad Request

Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.
20

  1. Inside the <VirtualHost _default_:443> block, add the directive ServerName myhostname.net .
    is the server only for nextcloud?

  2. I don’t understand 4443? https goes to 443.

Just check so I understand:

apache2ctl -S |grep DocumentRoot