The HSTS Header is just an extra layer of security. It is not suposed to redirect anything, it is supposed to block none HTTPS connections to a certain domain or subdomain. And of course this relies on external infrastructure and on your browser knowing that it has to block HTTP to a certain site. I can not explain in detail how it works, but thre are plenty of articles in the internet about that topic. Anyways…
A HSTS header and a redirect have to be in place if you want to be sure that users can connect through HTTPS without explicitly add HTTPS to the URL, or that no one will be able to connect with a plain HTTP connection. And if it isn’t possible, for whatever reason, to add the redirect and HSTS header directly to the configuration of your web server / VirtualHost, the only way to do it, is using the .htaccess file. And yes that means, that you have to check and maybe re-add these lines after an upgrade of your Nextcloud.