Should I worry about creating files via ssh if I ignore them on the client

ℹ️ Support
, , ,
1

I have no support/technical question and have seen the support category. (Be aware that direct support questions will be deleted.)

on

Which general topic do you have

Just recently Nextcloud completely erased many users and files from it’s /data directory. I have a feeling that the way I use it might be causing this problem (or other potential problems). I create web apps and use Nextcloud within my development and production workflow.

I self-host Nextcloud via docker on a NUC and use it as my production server for said web apps.

Development

I will use a git repo to pull down an instance of project and modify various unique files (like .env) to the project. This dev project folder is synced via the Nextcloud client on my Laptop

Source Files

I’ve set up the client to ignore such things as **/db/** and .env files as I usually deploy the apps with the database right beside it. This means I keep the source code in sync while bouncing between laptop and server. The ignore pattern allows me to have separate production .env files and ignored Postgres, MongoDB, etc files.

This workflow has been going no problem for over a year now, until now

Production

I’ll ssh into my production server (same NUC that runs Nextcloud) and cd into the mounted volume nextcloud/data/USERNAME/files/webapps/project_1 directory. There I will deploy the project_1 app via a docker-compose.yml script. The database files will be built right inside, for example webapps/project_1/db/DATABASE_FILES.

What To Do?

My worry is that this will cause problems, extra work for Nextcloud when it tries to scan files via occ or when restarting. Even though I ignore them on my client.

Is it nothing to worry about or should I be deploying any database files outside of the nextcloud/data folder?

Edit/Update

Looks like it was an issue with the log file become very large, filling up the drive, and sometimes unmounting the external drive (where the /data folder is)

nextcloud/data/nextcloud.log being over 100gb in size. I made sure the log level was set to only errors.

Looking at the logs revealed that there were permission issues caused by files generated by postgres database

UnexpectedValueException RecursiveDirectoryIterator::__construct(/var/www/html/data/nextcloud/files/webdev/WEBAPP/db/postgres): Failed to open directory: Permission denied

I realize it’s no bueno to create files outside of any nextcloud/webdav client and hopefully this will be a cautionary tail to any others attempting the same thing.

The Fix

clearing out the log files, remounting the drive, and restarting the container brought everything back online

echo "" | sudo tee /mnt/MYDRIVE/nextcloud/data/nextcloud.log > /dev/null
echo "" | sudo tee /mnt/MYDRIVE/nextcloud/data/flow.log > /dev/null
2

Hi @tawtaw

Is there a particular reason why you want to throw Nextcloud into the mix for this?

I’m not a developer so I’m not very familiar with the workflows involved, but I feel like you’re overcomplicating things here and that you’d be probably better off leaving Nextcloud out of the equation altogether.

Nextcloud is not a development or application distribution platform, but mainly a document sharing and groupware platform, and yes, if you change anything directly in the file structure or the files themselves on the Nextcloud server, you have to run occ files:scan to make Nextcloud aware of the changes, which is not really practical imho.

Maybe you could use the external storage app to avoid the file scans, but why bother, I mean you already have the files in a git repo, so why not just use git to keep them up to date on different machines and separate branches for development, testing, production, etc?

2 Likes
3

@tawtaw you agreed upon the matter that you don’t have a technical question.

so what’s this if not a technical question?

usually this would end-up in

but as there already is a n answer to your inquiry I’m gonna move this subject over to “support”.

please read and try to understand what you read next time.

4

I have a small addition. Nextcloud can do a lot but not everything. For example, Nextcloud still can’t make coffee. :coffee:

Nextcloud is a (web-based) collaboration software and uses the (for your idea really terrible) WebDAV protocol via HTTP/HTTPS. Nextcloud also prevents some file formats and file extensions, which is necessary, but doesn’t make it any better for you.

At the latest when you mention protocols such as ssh/sftp, git, … or backup, you can forget Nextcloud and thus WebDAV.

What you need on the client to synchronise the data from the server (hosted outside Nextcloud) is a Git client, sftp client or rsync.

If you use real Nextcloud functions, then web-based Nextcloud and WebDAV is the method of choice.

You never have to use “occ files:scan --all” to read in the data again if you use sftp only outside Nextcloud and inside Nextcloud the Nextcloud functions.

5

I think that the way Nextcloud automatically syncs files between locations is a friendly way to get started when not knowing about development tools too much.

As devnull pointed out, it really is not the right tool for development, even if it has worked for you for many years.

One risk, besides the unfortunate unknown event that made you lose data, is that the general direction of development for Nextcloud makes it incompatible with your workflow.

If you are developing software, It is really worthwhile to invest time in learning to use development tools!

1 Like
6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.