[solved] Server access fails after a while - Apache issue?

Hi all

For about 10 days (not able to be more precise), I cannot access to the nextcloud server after a few hours (not able to be more precise either…).
If I connect on the server machine itself and, on a web browser, type localhost/nextcloud (or any usual ip-server/nextcloud) I get the error message the server cannot be reached. All my nextcloud clients will also tell me they cannot access the server.
It works fine though just after restarting my machine and for some time.

The fresh install was made about 1 year ago and it has been working pretty well since then. Server version and system updates have been made as they came available.

Something weird : localhost always brings me to the apache default page of ubuntu, even if I cannot access to the nextcloud server. But (and here it is weird, I think), if I restart apache (sudo systemctl restart apache2), then, localhost/nextcloud will bring me to the nextcloud welcome page of my server…

I have no error reported when I look at the journal located on the Nextcloud admin account.

Beside system updates, the only thing I have done recently is to change my domain name. But it works fine during a while and even connecting on the server via localhost fails when the server cannot be accessed anymore. So, I don’t thing it is the root cause but I am not a professional.

Any ideas on where it might come from or on how to keep troubleshooting this ?

Thanks !

Kris

[/details]

Nextcloud 14.0.3

PHP
Version : 7.0.32
Memory Limit : 512 MB
Maximum execution time : 3600
Maximum upload size : 2 GB

DATABASE
Type : mysql
Version : 10.0.36
Size : 91.7 MB

Linux Mint 18.3 4.15.0-39-generic #42~16.04.1-Ubuntu SMP Wed Oct 24 17:09:54 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

sudo apachectl -V returns :

H00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Server version: Apache/2.4.18 (Ubuntu)
Server built:   2018-06-07T19:43:03
Server's Module Magic Number: 20120211:52
Server loaded:  APR 1.5.2, APR-UTIL 1.5.4
Compiled using: APR 1.5.2, APR-UTIL 1.5.4
Architecture:   64-bit
Server MPM:     prefork
  threaded:     no
    forked:     yes (variable process count)
Server compiled with....
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_SYSVSEM_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=256
 -D HTTPD_ROOT="/etc/apache2"
 -D SUEXEC_BIN="/usr/lib/apache2/suexec"
 -D DEFAULT_PIDLOG="/var/run/apache2.pid"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="mime.types"
 -D SERVER_CONFIG_FILE="apache2.conf"

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

<?php
$CONFIG = array (
  'instanceid' => '***************',
  'passwordsalt' => '***************',
  'secret' => '*************************',
  'trusted_domains' => 
  array (
    0 => 'localhost',
    1 => '192.168.x.x',
    2 => 'mydomain',
  ),
  'datadirectory' => '/srv/ncdata',
  'overwrite.cli.url' => 'http://mydomain/nextcloud',
  'dbtype' => 'mysql',
  'version' => '14.0.3.0',
  'dbname' => 'dbname',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'dbuser',
  'dbpassword' => '***************',
  'installed' => true,
  'mail_smtpmode' => 'smtp',
  'mail_smtpsecure' => 'ssl',
  'mail_from_address' => '***********',
  'mail_domain' => '**************',
  'mail_smtpauth' => 1,
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtphost' => '*********',
  'mail_smtpname' => '**************',
  'mail_smtppassword' => '***************',
  'mail_smtpport' => '465',
  'memcache.local' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'filelocking.enabled' => 'true',
  'redis' => 
  array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
    'timeout' => 0.0,
  ),
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 2,
);

The output of your Apache system log in /var/log/____:
I have a SSL warning but reaching https://mydomain/nextcloud works fine (when the server is up of course).

[Fri Nov 16 21:09:56.580878 2018] [ssl:warn] [pid 20062] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:09:56.606794 2018] [ssl:warn] [pid 20063] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:09:56.609142 2018] [mpm_prefork:notice] [pid 20063] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri Nov 16 21:09:56.609158 2018] [core:notice] [pid 20063] AH00094: Command line: '/usr/sbin/apache2'
[Fri Nov 16 21:12:29.115048 2018] [mpm_prefork:notice] [pid 20063] AH00169: caught SIGTERM, shutting down
[Fri Nov 16 21:12:33.288131 2018] [ssl:warn] [pid 20675] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:12:33.312653 2018] [ssl:warn] [pid 20676] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:12:33.315349 2018] [mpm_prefork:notice] [pid 20676] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri Nov 16 21:12:33.315367 2018] [core:notice] [pid 20676] AH00094: Command line: '/usr/sbin/apache2'
[Fri Nov 16 21:12:33.368416 2018] [mpm_prefork:notice] [pid 20676] AH00169: caught SIGTERM, shutting down
[Fri Nov 16 21:14:41.982446 2018] [ssl:warn] [pid 21233] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:14:42.007973 2018] [ssl:warn] [pid 21235] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:14:42.010385 2018] [mpm_prefork:notice] [pid 21235] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri Nov 16 21:14:42.010401 2018] [core:notice] [pid 21235] AH00094: Command line: '/usr/sbin/apache2'
[Fri Nov 16 21:41:24.474939 2018] [mpm_prefork:notice] [pid 21235] AH00169: caught SIGTERM, shutting down
[Fri Nov 16 21:41:25.569451 2018] [ssl:warn] [pid 25739] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:41:25.594604 2018] [ssl:warn] [pid 25740] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Fri Nov 16 21:41:25.597083 2018] [mpm_prefork:notice] [pid 25740] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Fri Nov 16 21:41:25.597098 2018] [core:notice] [pid 25740] AH00094: Command line: '/usr/sbin/apache2'
[Sat Nov 17 00:57:23.868815 2018] [mpm_prefork:notice] [pid 25740] AH00169: caught SIGTERM, shutting down
[Sat Nov 17 00:57:28.102948 2018] [ssl:warn] [pid 20255] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Sat Nov 17 00:57:28.127648 2018] [ssl:warn] [pid 20256] AH01909: 127.0.1.1:443:0 server certificate does NOT include an ID which matches the server name
[Sat Nov 17 00:57:28.130664 2018] [mpm_prefork:notice] [pid 20256] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
[Sat Nov 17 00:57:28.130679 2018] [core:notice] [pid 20256] AH00094: Command line: '/usr/sbin/apache2'
[Sat Nov 17 00:57:28.182975 2018] [mpm_prefork:notice] [pid 20256] AH00169: caught SIGTERM, shutting down

Output of my apache2.conf

# This is the main Apache server configuration file.  It contains the
#  ......... Introduction is cut to shorten the post  ...............
# Do NOT add a slash at the end of the directory path.
#
#ServerRoot "/etc/apache2"

#
# The accept serialization lock file MUST BE STORED ON A LOCAL DISK.
#
Mutex file:${APACHE_LOCK_DIR} default

#
# PidFile: The file in which the server should record its process
# identification number when it starts.
# This needs to be set in /etc/apache2/envvars
#
PidFile ${APACHE_PID_FILE}

#
# Timeout: The number of seconds before receives and sends time out.
#
Timeout 300

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100

#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 5


# These need to be set in /etc/apache2/envvars
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

#
# HostnameLookups: Log the names of clients or just their IP addresses
# e.g., www.apache.org (on) or 204.62.129.132 (off).
# The default is off because it'd be overall better for the net if people
# had to knowingly turn this feature on, since enabling it means that
# each client request will result in AT LEAST one lookup request to the
# nameserver.
#
HostnameLookups Off

# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog ${APACHE_LOG_DIR}/error.log

#
# LogLevel: Control the severity of messages logged to the error_log.
# Available values: trace8, ..., trace1, debug, info, notice, warn,
# error, crit, alert, emerg.
# It is also possible to configure the log level for particular modules, e.g.
# "LogLevel info ssl:warn"
#
LogLevel warn

# Include module configuration:
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf

# Include list of ports to listen on
Include ports.conf


# Sets the default security model of the Apache2 HTTPD server. It does
# not allow access to the root filesystem outside of /usr/share and /var/www.
# The former is used by web applications packaged in Debian,
# the latter may be used for local directories served by the web server. If
# your system is serving content from a sub-directory in /srv you must allow
# access here, or in any related virtual host.
<Directory />
	Options FollowSymLinks
	AllowOverride None
	Require all denied
</Directory>

<Directory /usr/share>
	AllowOverride None
	Require all granted
</Directory>

<Directory /var/www/>
	Options Indexes FollowSymLinks
	AllowOverride None
	Require all granted
</Directory>

#<Directory /srv/>
#	Options Indexes FollowSymLinks
#	AllowOverride None
#	Require all granted
#</Directory>




# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives.  See also the AllowOverride
# directive.
#
AccessFileName .htaccess

#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<FilesMatch "^\.ht">
	Require all denied
</FilesMatch>


#
# The following directives define some format nicknames for use with
# a CustomLog directive.
#
# These deviate from the Common Log Format definitions in that they use %O
# (the actual bytes sent including headers) instead of %b (the size of the
# requested file), because the latter makes it impossible to detect partial
# requests.
#
# Note that the use of %{X-Forwarded-For}i instead of %h is not recommended.
# Use mod_remoteip instead.
#
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

# Include of directories ignores editors' and dpkg's backup files,
# see README.Debian for details.

# Include generic snippets of statements
IncludeOptional conf-enabled/*.conf

# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

Output of my nextcloud.conf in /etc/apache2/sites-available

Alias /nextcloud "/var/www/nextcloud/"

<Directory /var/www/nextcloud/>
 Options +FollowSymlinks
 AllowOverride All

<IfModule mod_dav.c>
 Dav off
</IfModule>

SetEnv HOME /var/www/nextcloud
SetEnv HTTP_HOME /var/www/nextcloud

</Directory>

<IfModule mod_headers.c>
 Header always set Strict-Transport-Security "max-age=15768000; preload"
</IfModule>

Hi

No ideas ?

I can set a cron job to restart apache but it is better to understand what is broken (especially as apache is still up and running but it brings back the access to nextcloud !).

Thanks :wink:

You should start by looking at the apache logs when that happens. Also, how exactly do connect to your NC instance over the web? What is the hostname of your NC instance and does this hostname always resolve to the correct IP address?

Hi Budy

Apache log is attached to my first post and I have found a sigterm on the last line but I had this earlier on without leading to the issue. It is why I put Apache issue ? on the headline of my post but not sure as, when the issue occurs, Apache is still up and running. Entering localhost on a web browser on the server gives me the Apache welcome page of Ubuntu.

I connect to NC through clients and they all send a connection failure when the issue occurs. Then, I go directly on the server, open a web browser and get a connection failure.

Regarding the hostname, the clients use either the local network IP address or my website with https. On the server I try the 2 options plus localhost. When I can connect to NC all of that work. When the issue occurs, none of them works.

Thanks for your support !

Hi

I found the issue : to renew the certificate, certbot timer service stops apache. I changed my domain name recently and the old name was still in the list to be renewed. Certbot then was ending with an error and was not restarting apache.

For whom it could help :wink: