Adding the following to my Apache virtual host file fixed this for me:
Header edit Set-Cookie ^(.*)$ "$1;HttpOnly;Secure;SameSite=Strict"
You can find more info on setting secure cookies here:
Also, if you are changing header settings in your apache virtual host file you need to match the syntax used in the nextcloud .htaccess file to prevent duplicate header settings. For instance, if you wanted to set Header set Referrer-Policy "no-referrer" globally in apache you would need to change that to Header always set Referrer-Policy "no-referrer" because the always set condition is now used in the Nextcloud /var/www/nextcloud/.htaccess file.
You can check your current headers here: https://securityheaders.com/