I think this is a root cause: when something being saved on nextcloud external storage and this is not local folder, it could be downloaded to localhost first and then given to the user.
I do not have S3, but tried with WebDav and for me download started immediately: when I hit download, it starts like this External Webdav --> Nextcloud Server --> Client. You can see External Webdav dowload start as a green popup on external router. But you can’t see transmitted, because it was transmitted to the user via the Server in LAN and not external router.
The access.log is consistent with the one I reported in the main body of this thread at the top.
That is, propfind and get requests for the requested file for download.
Nothing appears at request time in the error.log.
My use of the external S3 storage in this issue was purely for test purposes, to see if behavior was consistent with what I’m experiencing when trying to download a file from local storage.
Are you suggesting that I entirely remove external storages from NC to see if problem is linked to these external storages ?
Thanks for the suggestion.
And sorry for the delay in my answer, couldn’t run tests earlier.
So I’ve tried to trace the process and got this thread of answers from the process :
0x00007fdff0868654 in ?? () from target:/usr/lib/apache2/modules/mod_http2.so
(gdb) bt #0 0x00007fdff0868654 in ?? () from target:/usr/lib/apache2/modules/mod_http2.so #1 0x00007fdff0884fa8 in ?? () from target:/usr/lib/apache2/modules/mod_http2.so #2 0x00007fdff08853dd in ?? () from target:/usr/lib/apache2/modules/mod_http2.so #3 0x0000559fc6482221 in ap_content_length_filter () #4 0x00007fdfef72b09b in ?? () from target:/usr/lib/apache2/modules/mod_proxy_fcgi.so #5 0x00007fdfef72c449 in ?? () from target:/usr/lib/apache2/modules/mod_proxy_fcgi.so #6 0x00007fdfef93abec in proxy_run_scheme_handler () from target:/usr/lib/apache2/modules/mod_proxy.so #7 0x00007fdfef93b9f9 in ?? () from target:/usr/lib/apache2/modules/mod_proxy.so #8 0x0000559fc6499910 in ap_run_handler () #9 0x0000559fc6499e8d in ap_invoke_handler () #10 0x0000559fc64b1d8b in ap_process_async_request () #11 0x0000559fc64b1f6e in ap_process_request () #12 0x00007fdff08849a3 in ?? () from target:/usr/lib/apache2/modules/mod_http2.so #13 0x0000559fc64a3280 in ap_run_process_connection () #14 0x00007fdff0885b8b in ?? () from target:/usr/lib/apache2/modules/mod_http2.so #15 0x00007fdff08893a6 in ?? () from target:/usr/lib/apache2/modules/mod_http2.so #16 0x00007fdff35686db in start_thread (arg=0x7fdfdc5b4700) at pthread_create.c:463 #17 0x00007fdff329188f in clone () at …/sysdeps/unix/sysv/linux/x86_64/clone.S:95
Triggering a download and then immediately cancelling it (just closing web browser of NC), still triggers the cpu spike of the same process for a few minutes (just no i/o activity because no download). With a read activity of the data drive. As if file was copied/loaded somewhere (temp location? Redis ? some cache ?).
Having seen a lot of mod_http2 in the process thread… I tried to disable http2
(a2dismod http2, and then a service restart for apache)
And BOOM : Downloads start immediatlely !!
(Also transfer rates were incomparably higher…)
So, what could be the problem here ? … getting closer though…
I have a feeling http2 is not the cause, just a symptom.
Could this issue be linked with apache MPM modules configuration ?
Unfortunately I never got to the last word on this one.
Having suffered a critical crash sometime after, I had to reinstall from scratch (I only backup the data and closely document all my install steps). (Which also gave me the opportunity to beef up the hardware a little).
But now, newer versions of NC, php, and etc all components, impossible to compare.
It turns out that after installation, the problem never reappeared…
So, in my current config, I’m running apache event mode and http2_module enabled.
Not a single issue like the one mentionned before… Sorry I can’t be of more help.