This is related to this thread: 503-Error / HPB service down / notify push can't connect to the Nextcloud server
As NCP and php had updates and I’ve more experiences I open a new thread.
Nextcloud version (eg, 20.0.5): 27.0.1.2
Operating system and version (eg, Ubuntu 20.04): Debian GNU/Linux 11 (bullseye) (aarch64)
Apache or nginx version (eg, Apache 2.4.25): Apache 2.4.56-1~deb11u2
PHP version (eg, 7.4): 8.1
The issue you are facing: on reboot notify push (sometimes) causes 503-Error / HPB service down
Is this the first time you’ve seen this error? (Y/N): N
NextcloudPi diagnostics
NextcloudPi version v1.52.4
NextcloudPi image NextcloudPi_06-02-23
OS Debian GNU/Linux 11. 6.1.21-v8+ (aarch64)
automount yes
USB devices sda
datadir /media/WD_500GB
data in SD no
data filesystem ext2/ext3
data disk usage 100G/458G
rootfs usage 6.0G/58G
swapfile /dev/zram3
dbdir /var/lib/mysql
Nextcloud check ok
Nextcloud version 27.0.1.2
HTTPD service up
PHP service up
MariaDB service up
Redis service up
HPB service up
Postfix service up
Internet check ok
Public IPv4 ***REMOVED SENSITIVE VALUE***
Public IPv6 ***REMOVED SENSITIVE VALUE***
Port 80 open
Port 443 open
IP ***REMOVED SENSITIVE VALUE***
Gateway ***REMOVED SENSITIVE VALUE***
Interface eth0
Certificates ***REMOVED SENSITIVE VALUE***
NAT loopback yes
Uptime 2:28
Nextcloud configuration
{
"system": {
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": {
"1": "192.168.1.104",
"4": "nextcloudpi.local",
"5": "",
"6": "nextcloudpi.lan",
"2": "https:\/\/***REMOVED SENSITIVE VALUE***",
"11": "109.250.64.197",
"14": "nextcloudpi",
"3": "nextcloudpi",
"20": "***REMOVED SENSITIVE VALUE***",
"21": "192.168.1.104",
"22": "localhost"
},
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "27.0.1.2",
"overwrite.cli.url": "https:\/\/nextcloudpi\/",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"instanceid": "***REMOVED SENSITIVE VALUE***",
"memcache.local": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"timeout": 0,
"password": "***REMOVED SENSITIVE VALUE***"
},
"tempdirectory": "\/media\/WD_500GB\/tmp",
"mail_smtpmode": "smtp",
"mail_smtpauthtype": "LOGIN",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"preview_max_x": "2048",
"preview_max_y": "2048",
"jpeg_quality": "60",
"overwriteprotocol": "https",
"loglevel": "2",
"log_type": "file",
"maintenance": false,
"theme": "",
"mail_sendmailmode": "smtp",
"mail_smtpsecure": "ssl",
"mail_smtpauth": 1,
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "465",
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
"updater.release.channel": "stable",
"data-fingerprint": "c43964c4e776c8cc78ea7cf4fb8f2ae1",
"logfile": "\/media\/WD_500GB\/nextcloud.log",
"app_install_overwrite": [
"admin_notifications"
],
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"htaccess.RewriteBase": "\/"
}
}
HTTPd logs
[Sun Aug 27 00:00:02.593984 2023] [ssl:warn] [pid 1078:tid 547970348096] AH01909: localhost:4443:0 server certificate does NOT include an ID which matches the server name
[Sun Aug 27 00:00:02.601319 2023] [mpm_event:notice] [pid 1078:tid 547970348096] AH00489: Apache/2.4.56 (Debian) OpenSSL/1.1.1n configured -- resuming normal operations
[Sun Aug 27 00:00:02.601397 2023] [core:notice] [pid 1078:tid 547970348096] AH00094: Command line: '/usr/sbin/apache2'
[Sun Aug 27 10:19:31.561483 2023] [mpm_event:notice] [pid 1078:tid 547970348096] AH00492: caught SIGWINCH, shutting down gracefully
[Sun Aug 27 10:19:48.809767 2023] [ssl:warn] [pid 1056:tid 547643728960] AH01909: localhost:4443:0 server certificate does NOT include an ID which matches the server name
[Sun Aug 27 10:19:48.867525 2023] [ssl:warn] [pid 1076:tid 547643728960] AH01909: localhost:4443:0 server certificate does NOT include an ID which matches the server name
[Sun Aug 27 10:19:48.874627 2023] [mpm_event:notice] [pid 1076:tid 547643728960] AH00489: Apache/2.4.56 (Debian) OpenSSL/1.1.1n configured -- resuming normal operations
[Sun Aug 27 10:19:48.874759 2023] [core:notice] [pid 1076:tid 547643728960] AH00094: Command line: '/usr/sbin/apache2'
Database logs
tail: cannot open '/var/log/mysql/*.log' for reading: No such file or directory
Nextcloud logs
```On reboot notify push sometimes (more than every second time) doesn’t start correctly, which causes an 503-error. HPB service is down.
/var/www/nextcloud/occ notify_push:self-test is:
âś“ redis is configured
âś“ push server is receiving redis messages
âś“ push server can load mount info from database
🗴 push server can’t connect to the Nextcloud server
Invalid response when getting test cookie
It seems like notify push somehow starts to early. I already changed /etc/systemd/system/multi-user.target.wants/notify_push.service to (this was a suggestion in 503-Error / HPB service down / notify push can't connect to the Nextcloud server - #13 by chrissi55):
[Unit]
Description = Push daemon for Nextcloud clients
After=apache2.service php8.1-fpm.service mysql.service redis.service
#After=mysql.service
#After=redis.service
Requires=redis.service[Service]
Environment=PORT=7867
Environment=NEXTCLOUD_URL=https://localhost
ExecStart=“/var/www/nextcloud/apps/notify_push/bin/aarch64/notify_push” --allow>
User=www-data
Restart=on-failure
RestartSec=20[Install]
WantedBy = multi-user.target
sudo systemctl restart php8.1-fpm doesn’t solve the problem.
When I run sudo ncp-config and choose clear-php-opcache in the “TOOLS”-section, everything runs well. /var/www/nextcloud/occ notify_push:self-test afterwards is:
âś“ redis is configured
âś“ push server is receiving redis messages
âś“ push server can load mount info from database
âś“ push server can connect to the Nextcloud server
âś“ push server is a trusted proxy
âś“ push server is running the same version as the app
This solution worked more than 20 times now.
I’d appreciate your feedback/ideas:
- Does anybody have the same problem?
- Are there maybe more changes to
/etc/systemd/system/multi-user.target.wants/notify_push.serviceneeded? - Does anybody know what commands
clear-php-opcacheinncp-configincludes? I couldn’t find anything about that. As a workaround, I would like to add that command to cron to schedule a reboot all x days.