Nextcloud 17.0.1 OS: Ubuntu 18.04.2 Apache version Apache/2.4.29 (Ubuntu) PHP version 7.2.19-0ubuntu0.18.04
I am still too much of a novice at this to know where to begin.
I have a local hard drive with ~700 GB of family photos, movies, and some music on it. I also have the Nextcloud server that is empty (except for what I’m trying to upload), and has a 1TB capacity. I want to “upload” the 700 GB of files from the local hard drive to my Nextcloud server.
I have been trying to do this since June of 2019 (five months).
I have used the desktop clients - 2.3, 2.5, and 2.6.
It runs fine for days, and then… It stops after uploading only about 590GB of the files. Then it just “refuses” to upload the rest, and gives me this meaningless message: “0 second(s) left, 0 B of 109 GB, file 0 of 3824”. (The number at the end of the message is somewhat different each time.) Then, since I don’t know which files were successfully uploaded, I have to start all over - I delete everything on the Nextcloud server, and start the upload process all over again. Only to go for several days, and then run into the same problem.
Right now I am at that point again: It refuses to upload the remaining ~110 GB of files, even though there is plenty of room on my Nextcloud server.
What is the location of your data directory on the server? It should be separated from your nextcloud installation directory. When this is done, you can transfer your mass data directly to the server into the user subdirectory of the data directory, and then
sudo -u www-data php occ files:scan user
Where user is your user id.
You can use rsync or scp to do the mass transfer. Make sure you change the owner and group of the transfered files to www-data.
My data’s top level directory is located within this directory: /mnt/ncdata/nextcloud1/files/
I don’t know a lot about this stuff, so I don’t know if that location is separate from the nextcloud installation directory. How do I verify that?
Please, what does this command do: sudo -u www-data php occ files:scan user ?
Tell me if I am understanding you correctly - You are saying that, using either rsync or scp, I can transfer the files, and then I should run the command sudo -u www-data php occ files:scan user (whatever that does), and then finally change the owner and group of all the directories and files I transferred - Do I have this right?
If I do what you suggest, I have a few more questions first:
Does this process you outlined also update the nextcloud database or whatever that indicates which files have been synced and which have not been?
If I follow your process, do I first need to delete off the nextcloud server the files that have already been successfully synced to it using the nextcloud desktop client? Or, are the two syncing methods “compatible”, and one will recognize syncing that has already been done by the other?
If I successfully do all this, then going forward, when I have just a very few incremental uploads to do to the nextcloud server, can I then revert back to relying on the nextcloud desktop client, and it will recognize the syncing that has already happened as a result of your process you are suggesting above?
I am unfamiliar with rsync, never used it. I am reading about it now, I see LOTS of options. Does this look like a good way to use rsync for my situation as I’ve described it:
How can it be that you don’t know where you installed your nextcloud??? You need to find that out, because you need to run the occ files:scan command inside that directory. It’s the same directory where you find your config/config.php file.
Remember you have to replace user by the user name you are actually using.
The method should be compatible to the upload via the web interface.
It must not be rsync. Use any method that works for you. scp or WinSCP would be fine as well.
The files:scan method does what you want: the files in your data directory are scanned and the database is updated. After that process, the state is the same as when you would have transfered the files via the web interface. That did not work for you, the reason is unclear. I proposed this method as an alternative (and it is much faster)
“How can it be that you don’t know where you installed your nextcloud???” Well, that question is easy to answer: First, because I am very new to all this, and I am stumbling across Nextcloud facts as I go along; and, because it was installed for me, I did not do the installation. (That person is not here, not sure how to contact him anymore.) I first learned of Nextcloud as being a server solution for the “ordinary person”, so I thought, Hey, that’s for me! Sadly, I am quickly learning that is not true. Oh well…
I followed your process. I got everything loaded onto the server, I got the owner and group changed to www-data, the permissions are 755 and 644. I did the files:scan. Everything looked good. It went quite fast, I was very happy - Thank you for your help! Then…
After ALL that was done, I had seven new family photos that I added to my local hard drive (my source). So, I wanted them to also be uploaded to my Nextcloud server. I realized that I could, in this case because there were only seven of them, easily add them via the web interface. But, I thought, I’ll try using the rsync command to do that. I wanted to do that because I thought, in the future, I could have a lot of files to add that are scattered all over many directories. I thought rsync would be smart enough to recognize that out of all my thousands of files I had just finished uploading, these seven additional files are brand new, so only they would get uploaded (into their proper directories). So, I ran the same rsync command as the one that worked earlier, but, instead, rsync refused to upload them: It keeps saying that I don’t have premission. But it is such a vague message, I don’t even know where it is saying I am having the permission problem: On my local drive, or on my Nextcloud server. And, why would I suddenly have a permission problem, when just minutes before it worked fine? I’m doing a lot of google searching now, I’m starting to think it was a mistake for me to follow the online stuff I saw to make the directory permissions on the Nextcloud server to be 755, I wonder if it should be 777, then I wouldn’t have this problem?
