After quite a bit more Googling, I was finally able to track down the issue…
Even though there wasn’t any errors logged on the server side, I did find errors in the Chrome console.
That eventually lead me to this post:
I had to set my WOPI access list to 0.0.0.0/0 instead of my subnet CIDR address.
After that, it’s working great!
It’s still not clear to me if this is a NC bug, having to open WOPI up like that. Also unclear as to what the security implications are, as I also use my Nextcloud server from the Internet.