These problems happen exactly if you don’t use Redis for locking. I can upload huge files using the web UI (10GB are no problem) on a Debian 9 system with a LAMP stack and Nextcloud 12.
The link above about transactional file locking and this howto should help you configure it: