Get files from the sftp server

i’m new to Nextcloud and I’m having some difficulties with structure of the folders the they add files.
I’m using openmediavault > docker > Nextcloud. I sync my work computer with SFTP and I would like to use Nextcloud at home. I want to use nextcloud like an repository, when I receive a call to make a change to a file to avoid making the sync by hand I would like to use Nextcloud to just download a specific folder and sync it while I’m working, then when I go to work I would like sync using SFTP between my home server. I sync from my work when the day ends and projects are wrapped up.
I already transferred the files to the nextcloud folder but none appeared.

Here an image to exemplify what I’m looking for:

Thanks in advance

The problem with sftp is that you go around Nextcloud’s internal file management. I see two options:

  • use the Nextcloud client (to sync) or connect through webdav (winscp supports it), so the connection goes through Nextcloud and it can track all files changes correctly.
  • use an external storage (which is where you connect with sftp), it must be outside the usual Nextcloud home folder. Advantage: you can still use sftp and for external storage Nextcloud allows changes by external programs and is tolerant to changes done by others.

Why do you use SFTP? For better performance?

At my home people use IPTV alot, and I being an designer I deal with big files and I save them many times bypassing losing them, I prefer sync in a specific time like lunch or at the end of the day and make it in one take and with shortest amount of time possible to avoid ruin my home network speed. I made a script using freefilesync that sync with my sftp then shuts the computer. Is possible to something like this with webdav?

dav2fs and rsync on Linux. Not sure about Windows what tools there are around for webdav, cyberduck, winscp, …

You could even consider the NC sync client, just stop syncing during you are working (to avoid constant uploads) and then start syncing when you are done.

Just FYI when you manually modify contents of the data folder, you have to run a file scan with occ to update the database or the files will not work correctly in Nextcloud. This is generally not advisable, but a lot of people seem to do it anyway…

A potential alternative might be using a separate SFTP server or using a separate folder on the server that isn’t in Nextcloud’s data, and then set that folder up as external storage in a Nextcloud.

I think I will use Winspc with WebDav I already tested and it does the job.
I’m asking this because I’m using docker, how can I force NC scan a directory?

I think for now I will go with the webDav is similar to SFTP

Found it

docker exec container_name sudo -u abc php7 /config/www/nextcloud/occ files:scan --all

Thank you for the support

WebDAV is too slow I think I will continue with SFTP and schedule an scan every time I sync.

It’s not recommended to temper with the data folder. I’d use external storage for that. With a good configuration you can get quite fast webdav, since database is involved, it won’t be faster than SFTP. But your setup is optimized and the NC code is good, it shouldn’t be faster via SFTP and doing a scan than directly using webdav.

What can I do to make that, by the way i’m using a Rpi3 not big of a deal machine.

I’d check first if your bottleneck is the bandwidth (few very large files), or the number of files (large number of very small files). The latter case is normally critical in Nextcloud and good database caching can improve the speed by several orders of magnitude. There are several topic here on the forum, some shared their configuration so you don’t have to start at zero.

This command will sync whole nextcloud, try to specify exact sftp/external solder instead to increase performance:

For me difference between --all and exact path is huge. It takes around 3 Hours to scan whole partition and only 6-10 min to rescan externals.

I obviously don’t know about your network set-up, but an idea might be to setup an SFTP server on your “Home CPU”, make a NextCloud share available to the SFTP server and then “drop” your files to it, using SFTP. The NextCloud client on “Home CPU” should notice the file changes and then sync to the NextCloud server.

How you would secure this or configure it in your firewall is beyond the scope of the question, but I think that’d work, and you would let NextCloud do its thing.

Will the client of NC be the SFTP client or the data will pass NC then my computer?

What I mean was that on “Home CPU”, you setup an SFTP server. Your “Home CPU” is already running the NC client. If you then connect to your “Home CPU” from the outside and transfer files to a folder that is watched by the NC client, it will detect that the file has appeared and once you’ve uploaded it, synchronize it with your NC server.

OTOH, you could also use an NC client at work. But I’m guessing that’s not possible since you’re already using SFTP.