Postgresql and overwritten files due to large parallel sync operations

Hello, I’m currently exploring the use of Postgresql instead of Mariadb for the first time in a Nextcloud deployment within Kubernetes. I’m employing the Nextcloud Helm chart along with a custom deployment of Postgresql through the operator available at GitHub - zalando/postgres-operator: Postgres operator creates and manages PostgreSQL clusters running in Kubernetes.

In this setup, I have a single Postgres replica proxied by pgBouncer, serving as the target database for the Nextcloud server.

I’ve experimented with both S3 as primary storage and the standard storage options. However, regardless of the storage method, I’ve encountered an issue when initiating the Nextcloud client (CLI or desktop) to sync a large folder (~200GB). The problem manifests as some files being stored with incorrect names. It appears as if too many parallel upload operations are causing overlaps in transactions, leading to scenarios where a small file (X) is initially written, a larger file (Y) overwrites file X, and as a result, file X becomes file Y. It ends up that after a forced resync, Nextcloud desktop recognizes a conflict and asks me what to do (either keep the local small file or the server version, another, larger, file that should have a different name).

Any insights or suggestions on addressing this behavior would be greatly appreciated!

So the Desktop client in both cases (assuming by CLI you mean nextcloudcmd not cURL or something)?

Sounds like a bug though if it was that easy to trigger I’d also expect it to have been reported previously unless it’s something new, but anything is possible!

Couple Ideas:

  • Check HTTP transaction logs (e.g. ingress web to see what’s really coming from the client)
  • Try to reproduce in web UI (mostly to confirm it’s the Desktop client)
  • Desktop client debug logs (these will probably be the most insightful since I’m currently suspecting it’s entirely a Desktop client matter, but that’s just my gut so could be wrong)

I doubt the choice of db is relevant, but anything is possible.