I was mistaken about the Raspberry Pi 4 cpu architecture - it does have the ARMv8 instruction set, it is only most debian based distros for the Pi 4 that still employ a 32 bit kernels.
I was able to solve my problems starting from scratch with the Raspberry Pi 4 minimal ARM edition of Manjaro. Porting over my existing nextcloud installation and data was a long and painful process (damn you silent errors) but finally it came around and now is running about as well as one could hope from such economic hardware.
For anyone facing problems on a similar setup I can only recommend to go through the official server tuning tips, all of them, step by step. Most impactful for me was to scale up the number of php-fpm child processes. When doing that it may also be advisable to raise the maximum simultaneous connections to your DBMS (in my case postgresql where 100 connections were exceeded in some circumstances (by a single user!) - resulting in an almost completely blocked system with maxed out RAM and swap)
In some places you may encounter the opinion that 64 bit kernel yields no practical benefits on a SBC like the Pi 4. I strongly disagree - it may be a flaw in nextcloud that it does not fully account for the limitations of 4byte integers in it’s software design, but this will only become more normal as time goes on - 64bit is the way to go.