Ncp: updated to 1.38 made instance unreachable and deleted config partial

Hi!

Just updated to 1.38 (odroid hc2 - running without problems for ages!)… and;

  • browser says unreachable (ip, url, doesn’t matter)
  • putty reaches ncp … but:
    → config letsencrypt deleted
    → ports closed
    → some other entrys not as before, but none “critical” (eg force https)

Any ideas?

= reboot … nohing changes
= load old config … nothing changes
= restore … well first time I didn’t backuped / made a snapshot sigh
= try getting a new cert … won’t work (syntax error in conf-file or “Connection refused”)
= trying reopen ports … doesn’t work “failed with code 403 (UnknownError)” - but router shows them open(!)

Any Idea?
Or do you need more info?

Rufio

<–! Paste this in GitHub report →

NextCloudPi diagnostics

NextCloudPi version  v1.38.0
NextCloudPi image    NextCloudPi_03-28-20
distribution         Armbian 20.05.0-trunk Buster \l
automount            yes
USB devices          sda
datadir              /media/myCloudDrive/ncdata
data in SD           no
data filesystem      btrfs
data disk usage      398G/1.9T
rootfs usage         25G/30G
swapfile             /var/swap
dbdir                /media/USBdrive/ncdatabase
Nextcloud check      ok
Nextcloud version    20.0.11.1
HTTPD service        down
PHP service          up
MariaDB service      up
Redis service        up
Postfix service      up
internet check       ok
port check 80        closed
port check 443       closed
IP                   ***REMOVED SENSITIVE VALUE***
gateway              ***REMOVED SENSITIVE VALUE***
interface            enx001e06374ce5
certificates         ***REMOVED SENSITIVE VALUE***
NAT loopback         no
uptime               14min

Nextcloud configuration

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": {
            "0": "localhost",
            "11": "2003:d0:c743:b000:c64f:6a6c:5cf2:bf9e",
            "1": "192.168.120.102",
            "5": "nextcloudpi.local",
            "7": "nextcloudpi",
            "8": "nextcloudpi.lan",
            "12": "dyn.dns.invalid",
            "3": "dyn.dns.invalid",
            "2": "https:\/\/dyn.dns.invalid",
            "20": "https:\/\/dyn.dns.invalid",
            "21": "192.168.120.102"
        },
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "20.0.11.1",
        "overwrite.cli.url": "https:\/\/dyn.dns.invalid\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "tempdirectory": "\/media\/myCloudDrive\/ncdata\/tmp",
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "overwriteprotocol": "https",
        "maintenance": false,
        "logfile": "\/media\/myCloudDrive\/ncdata\/nextcloud.log",
        "has_rebuilt_cache": true,
        "loglevel": "2",
        "theme": "",
        "mail_sendmailmode": "smtp",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "app_install_overwrite": [
            "previewgenerator"
        ],
        "log_type": "file",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25"
    }
}

root@nextcloudpi:~#

Enter configuration for letsencrypt
ACTIVE yes
DOMAIN dyn.dns.invalid
OTHER_DOMAIN
EMAIL mine@dns.invalid

Running letsencrypt
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for dyn.dns.invalid
Using the webroot path /var/www/nextcloud for all unmatched domains.
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. dyn.dns.invalid (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://dyn.dns.invalid/.well-known/acme-challenge/hb6jgg_akSVDWuBDHrRJPEE6dL2Nxiv0zFSGGMjGcOU: Connection refused
IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: dyn.dns.invalid
    Type: connection
    Detail: Fetching
    http://dyn.dns.invalid/.well-known/acme-challenge/hb6jgg_akSVDWuBDHrRJPEE6dL2Nxiv0zFSGGMjGcOU:
    Connection refused

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    your computer has a publicly routable IP address and that no
    firewalls are preventing the server from communicating with the
    client. If you’re using the webroot plugin, you should also verify
    that you are serving files from the webroot path you provided.
    Done. Press any key…

##############
Enter configuration for nc-forward-ports
HTTPSPORT 443
HTTPPORT 80

Running nc-forward-ports
upnpc : miniupnpc library test client, version 2.1.
(c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
desc: http://192.168.120.98:49000/igddesc.xml
st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.120.98:49000/igdupnp/control/WANIPConn1
Local LAN ip address : 192.168.120.102
UPNP_DeletePortMapping() failed with code : 403
upnpc : miniupnpc library test client, version 2.1.
(c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
desc: http://192.168.120.98:49000/igddesc.xml
st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.120.98:49000/igdupnp/control/WANIPConn1
Local LAN ip address : 192.168.120.102
UPNP_DeletePortMapping() failed with code : 403
upnpc : miniupnpc library test client, version 2.1.
(c) 2005-2018 Thomas Bernard.
Go to http://miniupnp.free.fr/ or https://miniupnp.tuxfamily.org/
for more information.
List of UPNP devices found on the network :
desc: http://192.168.120.98:49000/igddesc.xml
st: urn:schemas-upnp-org:device:InternetGatewayDevice:1

Found valid IGD : http://192.168.120.98:49000/igdupnp/control/WANIPConn1
Local LAN ip address : 192.168.120.102
ExternalIPAddress = REMOVED SENSITIVE VALUE
AddPortMapping(443, 443, 192.168.120.102) failed with code 403 (UnknownError)

Could not forward ports automatically.
Do it manually, or activate UPnP in your router and try again
Done. Press any key…
<–! Paste this in GitHub report →

your ports are closed, so LE won’t work. You can try to disable it in sudo ncp-config. Your Apache server is down, you could also look at /var/log/apache for clues

Just to clarify …

  • clean & running 1.37.2 (cert ok, ports open, nc-runs)
  • webinterface=>update to 1.38 “successful” / router not touched / reboot
  • unreachable 1.38.0 (cert empty, ports closed, nc-doesn’t run & apache down)

Well …
What do you mean with " try to disable it in sudo ncp-config"?

  • at “networking-letsencrypt” the entry is empty and there is no star.

And I’ve got no /var/log/apache … just /var/log/apache2 - with 4 empty 0-byte logs!

Greetings Rufio

PS:
Tried last hour to ““reopen”” the open (router says so) ports … nothing, system-info says “closed”!
Even a full reset of router, to simulate a new ncp-ip with closed ports, didn’t work!

From your router

  1. Create a static ip address for your nextcloudpi device by mac address.
  2. Port Forward 80 and 443 on that static ip from your router.
  3. Run let’s encrypt from ncp web or ssh.
    • Check your dyndns config is working if still having troubles.

run systemctl apache2 status, see if we learn something. Also try disabling letsencrypt from ncp-config

Well thats the way my ncp runs all time … i can reach it via ssh (with the assigned ip) but it is “unreachable” from browser or app!

nachoparker … please explain a little more - there is no switch ore whatsoever found by me.

<–! Paste this in GitHub report →
root@nextcloudpi:~# sudo systemctl status apache2
● apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2021-08-12 02:17:04 UTC; 38min ago
Docs: https://httpd.apache.org/docs/2.4/
Process: 885 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)

Aug 12 02:17:03 nextcloudpi systemd[1]: Starting The Apache HTTP Server…
Aug 12 02:17:04 nextcloudpi apachectl[885]: AH00526: Syntax error on line 5 of /etc/apache2/sites-enabled/ncp.conf:
Aug 12 02:17:04 nextcloudpi apachectl[885]: SSLCertificateFile: file ‘/etc/letsencrypt/live/my.dyns.invalid/fullchain.pem’ does not exist or is empty
Aug 12 02:17:04 nextcloudpi apachectl[885]: Action ‘start’ failed.
Aug 12 02:17:04 nextcloudpi apachectl[885]: The Apache error log may have more information.
Aug 12 02:17:04 nextcloudpi systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
Aug 12 02:17:04 nextcloudpi systemd[1]: apache2.service: Failed with result ‘exit-code’.
Aug 12 02:17:04 nextcloudpi systemd[1]: Failed to start The Apache HTTP Server.
root@nextcloudpi:~#
<–! Paste this in GitHub report →

Rufio (I’ll search for the log later)

Make sure you are in the latest version (type sudo ncp-update), then there should be an ACTIVE field under NETWORKING > letsencrypt, type no

OK - at least found the entry you meant nachoparker!
Sorry, my eyes “ignored” this new topline with setting first. %-O
But … it didn’t changed anything - I still can reach the ncp via ssh&ip, but not via browser&ip or browser&domainname.

So … a short “new” index;

  • DynDNS runs flawless
  • router untouched, ports open
  • updated nc-apps
  • ncp updated to 1.38
  • updated nc
  • updated nc-apps
  • deleted notifications inside nc, letsencrypt renewal error … used to for ages now
  • reboot
    (thats my normal "upgrade-way - most times rsync beforehand)
  • nc unreachable via browser&ip / browser&url
  • ncp unreachable via browser&ip
  • ncp reachable via ssh&ip
  • ncp deleted letsencrypt keys
  • ncp set wui to “false” (found that yesterday by reading all settings incl backup)
  • ncp changes are ignored
  • ncp/letsencrypt fail with posted messages
  • ncp filecopy by hand will be ignored/deleted
    => I tried to find out more about this “Aug 12 02:17:04 nextcloudpi apachectl[885]: SSLCertificateFile: file ‘/etc/letsencrypt/live/my.dyns.invalid/fullchain.pem’ does not exist or is empty” and the directory is absolutely empty … when copying the files to there, ncp deletes them at startup or when trying to actyvate/renew letsencrypt …

PS: “the Apache error log …” is still a zero byte empty file!

Rufio

Found one recurring Problem! AND My ““fix”” made my ncp available again …

I looked a bit for the infos from apache like;

  • AH00526: Syntax error on line 5 of /etc/apache2/sites-enabled/ncp.conf:
    = well, nothing to find
  • Syntax error on line 27 of /etc/apache2/sites-enabled/nextcloud.conf:
    = there was a strange entry
    #1-25 … “code” …
    #26 < / If Module >
    #27 Action ‘-t’ failed.
    #28 The Apache error log may have more information.

After delete of 27/28 … stop&start apache server … all fine again!
One reboot or letsencrypt renewal by hand … line 27/28 are back again with all problems.

Could this help you to find the gist of the matter?

Greetings RH

Yes that’s helpful. Thank you for the information. I’ll try to reproduce that and make the system more robust against that kind of issue.