I suspect this is more of a DNS and NAT problem than an SSL/TLS problem; I’ve got a similar issue with my home setup.
Long story short, your phone is probably talking to a different web server (maybe your router’s admin web page?) when it tries to connect to Nextcloud from within your LAN. You can fix this by adjusting DNS settings for your phone when connected to the LAN, or by messing with your NAT/port-forwarding rules, if your router gives you deep access to those.
Here’s a breakdown of the problem I suspect you’re seeing. A little wordy, but hopefully it helps anyone else with similar trouble.
Happy Path: Remote
When your phone is out and about and it looks up your Nextcloud domain’s IP address from any public DNS server, it gets the public IP of your home router. You’ve set up your router to port-forward connections coming from the internet through to your Nextcloud installation’s LAN IP, so the router does some NAT magic and it all just works.
Happy Path: Local
When your home computer needs your Nextcloud domain’s IP, I suspect it asks your router or some other DNS server you’ve got running, or maybe you’ve got an entry in your hosts file. Regardless of the method, the computer probably receives your Nextcloud installation’s LAN IP, so it connects directly - no port forwarding, no NAT, no problems.
Unhappy Path: Mixed
When your phone is on your home network and looks up that IP, it’s probably getting your router’s public IP address, and it starts a connection to that IP just like it would if you were on some other network.
But since your connection isn’t coming in from the internet, your router doesn’t apply the port-forwarding rule. Without it, your router just sees a web connection request coming from your LAN, and addressed to an IP that belongs to the router. It doesn’t matter that it’s the router’s internet-facing IP, just that the connection comes from the LAN and is addressed to the router, so what the router is supposed to do is… connect you to its web admin console.
Your phone checks the certificate provided by the admin console, finds that it’s NOT your Nextcloud server’s certificate, and drops the whole connection attempt with “SSL initialization failed”.
So, how to make sure your phone gets your Nextcloud’s LAN IP, when it’s connected to the LAN?
- Make sure your local DNS server (your router?) is giving out the right IP when queried for your Nextcloud domain name
- Make sure your phone is actually set to use your local DNS server, which isn’t as obvious as it should be.
- If you’ve been connected to your Nextcloud through another internet connection pretty recently, it may have the IP cached instead of asking again.
- If you’ve still got your cellular data on, your phone might be using its DNS settings for that connection.
- Android’s “Secure DNS” feature ignores the DNS server hints provided by DHCP to use Google’s service, unless you’ve set up your local DNS server to do DNS over SSL, or just disabled the feature.
- If your DHCP server provides multiple DNS server hints, your phone might just pick the wrong one. Mine does, because my local DNS resolver is sometimes slow.
Since you’ve got multiple subnets, you will also need to be sure that your Nextcloud server is reachable from the subnet you connect your phone to - whether you just allow web traffic between the subnets, or you have some additional port-forwarding rule with another internal address, that’s another layer to check for further issues.