Sure you could use a self signed certificate. But that gives you and your users certificate warnings in browsers and client apps. In general it is best practice to use signed SSL certificats like the ones from Let’s encrypt nowdays. Especially when it comes to a service like Nextcloud, with which you probably want to use files with many diffrent apps and probably share things with other people too.
In general you will have less problems and annoyances with a proper signed certificate. The best example is Chrome, which no longer allows to save exceptions for self-signed certificates. You would have then to click on “Allow” every time you want to visit your own cloud.
And I honestly don’t see any reason why you shouldn’t use a domain name. If costs are a concern… You can get a Domain name for less than 10$/year.