Downloading 900 GB file ruins docker image - random internal server error 500

I had exported my data via the new “data migration” app that is available for nextcloud 24. A little bit more than 900 GB. Then I logged in to the webclient and downloaded the file, with firefox if that matters. At some point it stopped downloading and the nextcloud instance I was downloading from was showing random “internal server error 500”. Random means, that reloading the page would sometimes work, sometimes only for half the page, sometimes it would show the web server’s error message, sometimes the nextcloud error message.

I could not find the issue, rebooting did not help. I noticed with iotop that the mariadb process was writing 20-30 MB/s to the disk, even after rebooting. But the error log did not show any hints (at least to me).

The only thing that helped was to recreate the docker image.

I thought it was a one off and started to download the file again. Same issue after a few hundred GB. Again only recreating the docker image solved the issue.

So in case somebody else comes across the same issue, I hope this helps to “solve” it - and maybe that person is better at debugging it and can file an issue with useful information against the nextcloud server.

Why not use the tooling available on :4443 or ncp-config to backup and restore ncp?

Did that when moving to docker and now I have the issue that the dbpassword changes on every reboot. Thus I wanted to test something which does not create a backup of everything but just an export of one’s user data.

Moreover, this would not solve the problem but just work around something which seems a serious issue, even though I cannot debug it.

1 Like

I’m confused on what the issue is and also how you are addressing it. Would be nice to understand what is happening and put together a plan for possibly addressing it moving forward.

Downloading a 900 GB file from a docker NCP instance fails after some hundred GB and causes that instance to have random Internal server errors (500) which cannot be fixed by rebooting. Only recreating the image helps.

I cannot debug it but I do not need any help either since I found a workaround. I just wrote this so that others find the “solution”.