Importing Many Files and Directories Fails

🚧 Installation
1

I have many files (20,000+) totaling about 300GB of disk space and am trying to add them to NextCloud. I’ve updated my php.ini with:

upload_max_filesize = 512G
post_max_size = 512G
max_input_time = 28800
max_execution_time = 28800

And when I drag the top-level folder into NextCloud, it seems to silently ignore the drag/drop despite changing to the blue color that indicated I dragged it to the correct spot.

What I really wish I could do is some operation at the Linux command line that would import all of my files. Does such a thing exist?

I have separately tried to install the Linux client; however, for CentOS 7.9 the AppImage client file fails with dependency problems. I’ve created a separate topic for that, but my point here is that I can’t just synchronize to a folder because the client isn’t working.

I have been successful in dragging smaller collections of sub-directories via the GUI, but this is going to get very tedious. I’ve been doing just that for the past 3 hours. Hence, my hope that someone has a better way to import large numbers of files and directories into NextCloud. Thanks for any assistance you might offer!

Regards,
Steve Amerige

2

I successfully imported 11440 directories and 108036 files by first copying them into a location under my data directory, fixed the modes, then used the console.php that has the files:scan functionality that brings the files into Nextcloud’s awareness:

# logged in as the user that Nextcloud is running as
SRC='/source/from/top'   # Top-level directory containing 11K dirs and 108K files
NCWEBROOT='/dest/to/web'
DST="$NCWEBROOT/data/admin/files/top"

# Copying with sudo to because $SRC files have a different owner than $NCWEBROOT
# Used the --delete option to ensure absolute equivalence
sudo rsync -az --delete "$SRC/"  "$DST/"   # Ending / is necessary to copy directory contents

# Fix up the owner to the one that Nextcloud is expecting
chown --reference="$NCWEBROOT" -R "$DST" 

# Let Nextcloud become aware of the changes made outside the GUI
php "$NCWEBROOT/console.php" files:scan --all -n > "$NCWEBROOT/scan.log" 2>&1

The process took hours, but saved me days of error-prone manual steps!