Removing index.php from URL doesn't seem to work for me

Did you also try the official documentation? Installation on Linux — Nextcloud 34 Administration Manual

In general, there should be no need to edit .htaccess manually, and you probybly don’t need to set htaccess.IgnoreFrontController => true, either, unless one of the conditions described here applies to your setup: Configuration Parameters — Nextcloud 34 Administration Manual

In most cases you just need to enable mod_env and mod_rewrite, and add the following to your config.php:

For an installation in the web root:

'overwrite.cli.url' => 'https://cloud.yourdomain.tld/',
'htaccess.RewriteBase' => '/',

Or, if Nextcloud is installed in a subdirectory of the web root:

'overwrite.cli.url' => 'https://cloud.yourdomain.tld/nextcloud',
'htaccess.RewriteBase' => '/nextcloud',

Then run:

occ maintenance:update:htaccess

If that doesn’t work, please provide more information about your setup, ideally by filling out the Support Template.