Hi I’m using NextCloud (NextCloudPi 1.50.2 - NC 24.0.5) installed on Rasperry Pi4 with Docker and Portainer.
I specify that it is a clean installation as per the NextCloudPi image and without other services or containers installed in addition to the base of the nextcloud image as downloaded. No other custom configurations.
When I try to download a file with IDM (Internet Download Manager) as soon as the download starts, the NextCloud GUI remains blocked for all users for a long time. If I download the files without IDM everything works perfectly. Is it possible that the block is caused by the overload of the multiple connections that IDM creates to download the file? … is there a way to set the NextCloud server to prevent or limit multiple connections on the same file?
I tried to change several php-fpm configuration parameters:
pm = static
pm.max_children = 100
pm.start_servers = 12
pm.min_spare_servers = 6
pm.max_spare_servers = 18
but the problem has not been solved.
At this point I think it’s a NextCloudPi bug or a wrong configuration of the NextCloud server that I can’t understand.
PS: it is impossible that it is the internet connection since I have a 1000 Mbit/s professional fiber connection with guaranteed minimum bandwidth and with other connected servers that manage hundreds of simultaneous connections … the same RasberryPi 4 on which it is also installed NextCloud with the FTP service is able to transfer files at a speed of 50-60MByte/s … so I would say that the problem only concerns NextCloud …
I must clarify that I am not an advanced user of Linux!
There are no particular reports on the log but only a temporary block of the NextCloud GUI during the download with IDM. The strange thing is that even immediately interrupting the download from IDM, the GUI hang still persists for several minutes and then everything goes back to normal.
IDM by default downloads with 8 parallel threads if you don’t change the default settings. Even setting less than 8 threads the problem still occurs. It seems that the problem occurs as soon as IDM starts negotiating with the server regardless of the number of threads.
What about your load average during the download? Check with htop or equivalent. I’m curious if this IDM is doing by something like slamming your Pi with download requests.
During normal downloads (without IDM) everything works perfectly and the system is able to manage even more simultaneous downloads from different users … but if only one download is made with IDM, the Nextcloud GUI freezes and remains the same for all the others users (including Admin) for several minutes before unlocking after the download is complete. If the download takes a long time for large files the GUI is blocked for all that time. If you interrupt the IDM download the GUI will unlock after a few minutes.
At first glance the resulting load during normal downloads doesn’t appear to be much different than with IDM.
I will try to install the htop command and post the results.
Tanks.
Ok, so what I see there is that the IDM download is nearly maxing out your Pi’s CPU (3.91 out of 4 max load avg). It’s not clear whether that’s the issue but it could be a contributor at least. I would expect it to clear up almost immediately when the download stops if that was the entire issue.
Do you have the Nextcloud brute force app installed, or anything similar for Apache that does rate-limiting?
Actually the problem is that despite interrupting the IDM download the GUI remains blocked for all users for several minutes after closing IDM and this doesn’t seem normal!
I installed the original NextCloudPi image in a docker container with portainer without any further modifications. The only changes I’ve tried are the ones I posted above to the php-fpm in an attempt to fix the problem which remained the same anyway.
PS: Another strange thing is that if I start the download with IDM and interrupt it immediately before it starts the GUI still hangs for several minutes even if the download hasn’t started. There seems to be a problem while IDM is negotiating the request to NextCloud.
I performed other tests and I noticed that during the download with IDM the transfer speed is always high and if at the same time I perform other downloads with the FTP service from the same device (Raspberry) everything works perfectly, both the download with FTP and the one with IDM they proceed in parallel at the maximum possible speed without any system blockage … the only thing that gets stuck is only the Nextcloud GUI interface.
For this reason I am increasingly convinced that it is some specific Nextcloud BUG with IDM.
Everything else works smoothly even when transferring files using other services at the same time. The Raspberry never crashes… only the NextCloud GUI crashes.
Tanks
i think this https://www.internetdownloadmanager.com/features2.html looks like a rather nasty piece of software; it probably opens as many connections as possible and might send requests (range) that your server cannot handle. if you close the software on the client-side, the open connections on the server remain in TIME_WAIT for some time. ie are still blocked.
check with netstat when you are connected to the server with this program.
trying to read one file in many separate small sections at the same time might cause i/o-issues (esp on a raspi with additional layers like docker). this might show in the webserverlogs or you might be able to see sth with iotop.
if your network-connections are fast enough maybe you do not need this software; or, if time is really an issue, you could use other protocols to access those files (be carefiul with the permissions, though.)
(also, ftp and http cannot be compared in terms of file-transfer, since one is optimized for many parallel transfers while the other one is not. )
GOOD LUCK!
In fact, i don’t use that idm software but some users of my nextcloud use it and when they connect to the cloud and download something they block the whole system for the other users too … i think a block in the NextCloud server configuration would be needed for the too many connections generated by IDM otherwise the system will crash.
Tanks
)