Permissions issues on local external folders


#1

Sorry to be making another support topic but I though tit was important to keep this separate from other issues threads I had made for the sake of clarity.

My setup:

Nextcloud version: 13
Operating system and version: Ubuntu 18.04 and Windows 10
Apache
PHP version: 7.1 I believe

image
In summary:

  • I have a windows laptop.
  • I have an Ubuntu desktop.
  • The desktop is running the server.
  • The desktop has two local external folders mounted to the NC server.
    1. The Archive which is a 3TB drive used for long term storage and large media files.
    2. my work folder which is physically located at /home/david/work in my user home folder.
  • The work folder is synced to my laptop using the NextCloud client.
  • The Archive is mounted as a webdav network folder giving me access to the 3TBs of files if I need to get something off my harddrive and back into the archive, or in case I need to pull a file out of the archive onto my Laptop to work with.

My Problem

When I edit a file in my work folder on my laptop (which is located at C:\Users\david\work) the changes will be shot up to NC through the sync client. Those changes will be then be made to the file by NC under the user www-data to the twin file at /home/david/work.

When it does this, I find a file in my work folder that I, as a user, cannot edit, delete, rename, or move, because the owner is www-data. I can use sudo or change the file’s owner, but if I add another file to the folder on the laptop, it will have the same problem, so it’s not a very good longterm solution.

The opposite is also true. If I directly edit a file locally in /home/david/work, NextCloud has trouble adding it to it’s database because it isn’t owner of it. I get a bunch of sync errors from both sides, saying things like “cannot access that file or directory” or “you lack the necessary permissions to upload or create files here”

I tried adding myself to the usergroup www-data, and I tried adding www-data to the usergroup david, but by default it seems that only the owner of www-data has the ability to edit or delete files and putting myself or www-data into either of those categories does not work.

I have already checked Documentation and the forums.

But I can not seem to find an answer.

How can I fix this?


#2

This is the important part for fab’s fix, but I’m pretty sure it doesn’t apply to your use-case. These are NTFS-specific options to mount his volume with the www-data user and group as owners. The umask 002 part allows group members to modify the files, but that’s not something you want to specify on your whole / (or even /home) volume.

It would seem that what you want is for group-members to be able to write to these files/folders. For a one-off solution,chmod --recursive g+rw /home/david/work should allow the www-data user (since it’s a member of your ‘group’) to do this, assuming Apache and AppArmor aren’t involved.
As an ongoing thing, new files or folders created within this directory will have different ownership and different permissions. I can suggest two options:

  • Hacky solution: also run chown/chmod when you run your files:scan, so these things are corrected quickly
  • Proper solution: Take a dive and look into how you can make sure these new files get created with the correct permissions. You might want to start with “Understanding umask Value Usage”, but I’ve never gone deep enough to give you the gist. Perhaps someone else has?

#3

Okay yeah the umask thing sounds like what I need to do.

I typed umask 003 in command line and then I mkdir a test directory. I checked it’s permissions and they look like what I intended.

But if I create a new folder inside of Nautilus etc, it does not have the correct permissions. I am guessing that the umask for a user in CLI is different than when they are using Nautilus. In the case of Nautlius, the user is actually nautilus not david? And that means that in order to change the umask I need to change it for www-data.

Also having read that stuff I now understand your first statement about 002. Why wouldn’t I want to give umask to my work folder on my home volume? It would only open it up to attack from people within the groups, so www-data and david, and I’m pretty confident about those being safe… right? Why shouldn’t I make it so the folders are mounted with the www-data user and groups?


#4

Pretty much your options as far as umask goes are:

  1. Suggest a umask for all programs acting on behalf of a given user (or two users, in your case), but the programs have the option to override it.
  2. Enforce the umask on an entire given volume, because the filesystem of that volume doesn’t support Unix permissions

Nautilus is acting on your behalf and running as your user account (david), but the umask permissions are only a suggestion/default, which Nautilus may choose to ignore. I’d also make sure you’ve entirely logged out and back in, in case Nautilus spawned from a process running before you set your umask, and didn’t think to check. If Nautilus continues to ignore the umask after that, then the site I linked suggests cleaning up with chmod as needed.
That said, yes, you would also need to set the umask for the www-data user so that (hopefully) Nextcloud would follow that default/suggestion of permissions for anything it created. If Nextcloud and Nautilus obey your umask suggestion, this should work smoothly, and you won’t have to worry about the stuff I mention next.

You can’t only set it for a given folder within a volume - it would have to be set for the whole volume, to my knowledge.
The command you used gives all programs running as david a suggested set of permissions to use when creating files. That’s option 1.

Option 2: The mount option they used gives the Linux kernel a set of permissions it should pretend the NTFS volume in question has, since NTFS doesn’t support Unix-style permissions. If you want to create a new partition of a type that doesn’t support Unix permissions (any such filesystem is listed listed with a umask option in man mount), then mount it as your /home/david/work folder, that could work. I think you’ll likely get more mileage out of using chmod on anything that appears with the wrong permissions, or finding a file manager that respects your umask settings (assuming Nautilus is the only thing ignoring your umask command, and continues to ignore it after a logout or reboot).


#5

So doing some more research I’ve come to similar conclusions as you. I need to change the usmask for www-data. From my reading it seems that I can change my own umask fairly easily by changing my .bashrc or .profile to include a command of umask 003 The problem is… I can’t find out how to change the umask for www-data. All the pages I find talk about it adding the umask command to the startup scripts etc for apache2 or checking under /home/www-data etc, but none of the those files that they suggest changing actually exist. I think I understand that what each of them is suggesting is to change the equivlant of the .bashrc for www-data but again, that doesn’t do me much good if I can’t find the .profile file or envars for apache2 etc.


#6

Found a couple of ServerFault questions, in the hopes it helps.


This one’s got a ton of things the OP tried, which might actually be helpful. Make sure you do the full service apache2 stop then start, rather than just a reload when you try these out.

The first answer from this really similar question includes a way to test to make sure you’ve applied it correctly to the Apache server process. That could be handy in finding out if Nextcloud continues to ignore your umask even after it’s applied properly to Apache.


#7

Yeah that’s the thread I was looking at but I don’t have an envars folder in apache etc. Also I’m running nginx anyway.

but I did find /etc/profile/nginx once I actually went to the command line. I couldn’t see profile even with hidden folders set to visible for some reason in nautilus, but once I tried in CLI I found it. I added umask there. and I’ve stopped and started the server and I’m fixing to check and see if it worked.

As for the script for checking if stuff works… uh… I actually don’t know how to run that script?EDIT…Figured out how to do the php script. Save it as a file and then run it using php- f script.php

Anyway I’m having to shift some stuff around because my folders got kind of messy in the process of trying to back stuff up. I don’t think I have the group shared permissions yet the way I want, but I’m going to wait till after this copy and sync process finishes.

I also found out how to remove the file types limitations that I was having trouble with. I edit the file sync-excluded.lst which was under the applications folder for the NextCloud Client, and on Windows was in the Program Files for NextCloud.

I removed a bunch of limitations for temp files and stuff like *_.* type stuff.


#8

Hey @mactrent So I fiddled around with the umask thing, and I may have got it working, or may have not, but I think I’m running into a problem on the opposite end of the equation.

When I create a file on my Windows laptop, it can’t sync it up to the cloud, saying that it doesn’t have permission to create subfolders. I don’t know what that means though. But from what I can tell looking up that specific wording it seems to be an issue with the client, not an issue with the permissions set on the local external folders of my desktop server. If it were just that then the client should be able to at least push the folders up to the web GUI etc.


#9

Hmm, and are you able to create files from the web GUI then? If there is (or was) an actual permissions issue on the server side, the web GUI will usually show a warning “you can’t create files and folders here” or similar. If you fixed the permissions recently, you might need to do an occ files:scan before Nextcloud realizes it’s fixed.


#10

So I recreated all the folders from scratch and I think I figured out that issue though not sure how to fix it.

I can create a folder in Work on my desktop and it will be visible in the web interface and will sync down onto the laptop. But I can’t add any folders to that folder from the web interface, and if I put a folder in it on my laptop, the sync client will refuse to let it through saying I don’t have permission.

If I create a folder on web GUI, it will appear on my desktop folder, but it will have a little lock symbol on it. It’s owner will be www-data.

If I create a folder on my laptop it will sync up to the web GUI via the client, and then appear on my desktop but also be impossible for me to edit and the owner will be www-data.

My Resources folders etc are able to be accessed etc by webdav because I had previously chmod 775 them. But I imagine I would have the same problems that I’m having with the Work folder if I added folders to them through the Web GUI etc.

I can’t chmod things every single time I create a folder, obviously, so this really just brings us back to the original problem of umask etc. All these different variations of behavior are based on the default way that www-data grants permissions on new files it creates. So I need to fix that. I will try again with the instructions you linked and see if now that I have a better understanding of the process, if I can’t perhaps fix this. I just need to be more systematic.


#11

Hey I think I got everything fixed. I’m fixing to do a reboot to double check but I wanted to log some quick notes here for anyone else who might have this or a similar problem in the future.

In order to do this I first added david to the www-data usegroup and added www-data to the david usergroup.
This might be a little bit of a security issue but I really don’t think it’s that major.

Secondly I needed to change the umask for www-data, so that when it created a new file in the folder it would do so with a 775 permission (meaning that user and groups would have full write, read, execute access).

This can be done on my nginx server by doing the thing describerd here: https://stackoverflow.com/questions/11584021/nginx-php-fpm-umask-setting/21318057#21318057

if you use systemd [i.e. Ubuntu 16.04], then edit /lib/systemd/system/php5-fpm.service
And edit chapter “Service”:

[Service]
UMask=0002

In my case I edited php-fpm 7.2 because that’s the version I’m using.

Okay so now www-data will create new folders on the system that can be opened by the groups etc.

But the next thing to do is to make it so that files made by the user are editable to the www-data. The way to do that is to change the umask for david. It’s enough to just add the line umask 002 somewhere in your .bashrc or .profile file. If you want to you can also add it to your root user account in a similar way.

This still isnt’ enough though because as mactrent said, nautilus etc are going to have their own default umasks to use when you create folders etc with them. According to this post it is possible like this: https://unix.stackexchange.com/questions/254378/how-to-set-umask-for-the-entire-gnome-session

up vote 2 down vote

The problem is that mentioned by Sebasth. I tried many things, but then i found a workaround which consists in overwriting the (per-user) UMask of dbus:

$ systemctl --user edit dbus

In the file that gets opened, just write:

[Service]
UMask=002 # This is the umask i want to use

The file gets saved in .config/systemd/user/dbus.service.d/override.conf and overrides the dbus default umask, which i presume is inherited from systemd --user, since dbus is launched by it. Just logout and login again and gnome applications should use the specified umask. It works form me.

I’m going to restart my machine and see if it works but I think it should.


#12

Bad news, it does not seem that the changes meant to force nautlius to recognize the umask 0002 are working. I also tried going into /etc/login.defs which has some kind of setting for changing group umask defaults, though the wording of the instructions is a little hard to parse.

I’ve tried changing multiple different files and nothing seems to work.

Is there a way that I can make nextcloud write as the user david instead of www-data?


#13

Confirm the following:

1. You have applied the workaround to dbus.service. Please check as follows:

systemctl --user cat dbus | grep UMask=002

2. After applying the workaround, you have completely logged out your user before logging back in. The simplest way to ensure this is by rebooting (or you can have checked loginctl from a root login).

The point is you need to stop/restart the user dbus daemon. It is a systemd user service. If you left even a non-GUI login, OR you had enabled systemd user “lingering”, the dbus daemon will not be restarted. I guess you could login on the console and run systemctl --user restart dbus, this will do bad things if you have a GUI login at the same time.

Then, you can check the following:

Umask of dbus:

eval "$(systemctl --user show -p MainPID dbus)"
grep Umask /proc/$MainPID/status

Umask of nautilus:

nautilus&
PID=$(pgrep -u "$USER" nautilus)
grep Umask /proc/$PID/status

I also assume you use a normal GUI login from the GUI login screen, and do not use startx from a text login. I’m not exactly sure what happens in that case, because it works differently. You seem to end up with two different user dbus daemons :-).


#14

But before that, you need to make sure the group owner of the created files is always “www-data”. Did you check that? It is not enough to add “www-data” as one of the secondary groups of the user David. In addition, you must run “chgrp -R www-data /home/david/work”, and then "chmod g+s -R “/home/david/work” to set the set-groupID bit.

I’m using “bindfs” myself instead of all this. It avoids the special case that moving files into the shared directory will leave them with the “wrong” permissions.


#15

bindfs example line for /etc/fstab: https://unix.stackexchange.com/a/278399/29483


#16

Okay so I might not be understanding the technology very well but it sounds like to use the bindfs technique I have to use the Fuse filesystem which means I can’t use my current NTFS (for my archive drive) or Ubuntu Logical Volume (for my SSD that the OS runs on) ?

To do that solution I’d basically need to reinstall linux etc right?

Not having the ability to copy in files to the directory with correct permissions would not be ideal, but I’m also so far along in this process I don’t know if I’m really up to restart everything and use a different filesystem that would be unreadable to my Windows partition. Maybe I’m misunderstanding the ideas though.

root@lovelace:~# systemctl --user cat dbus | grep UMask=002
Failed to connect to bus: No such file or directory

I can confirm that each time I’ve tried this stuff etc I’ve completely rebooted to make sure it restarted the stuff correctly.

Further results requested:

root@lovelace:~# eval “$(systemctl --user show -p MainPID dbus)”
Failed to connect to bus: No such file or directory
“”: command not found
root@lovelace:~# grep Umask /proc/$MainPID/status
grep: /proc//status: No such file or directory

and

root@lovelace:~# PID=pgrep -u “$USER” nautilus
-u: command not found
root@lovelace:~# grep Umask /proc/$PID/status
grep: /proc//status: No such file or directory

#17

bindfs is a stacking filesystem, not an alternative to a real one. Notice the source is not a block device, but a directory.
Note, with systemd, you cannot stack bindfs over the root directory of another mount point. You have to mount bindfs on a subdirectory of the real filesystem.

I made a mistake in the last set of commands. It was missing $() around $(pgrep -u “$USER” nautilus). I fixed it now.

If you want to run nautilus as root, I cannot help you. This is not supported and I have no experience with it. Otherwise: all the “systemctl --user” commands are to be run as your normal user. They refer to the per-user service manager, not to the system manager.


#18

Hmm, with NTFS mounted with gid=www-data, that should take care of the group part AFAIK. So at least no need to worry about “chmod g+s -R”. I don’t know if it has an alternative to bindfs “perms=g+wX”, though I think there is for vfat, so there’s a good chance. U&L stackexchange might have an example


#19

Hmm, I dunno. It looks like your NTFS options should have set this up already, though the docs on ntfs-3g permissions make my head hurt.


#20

Okay so would I then basically just turn my Work directory into a bindfs filesystem? Sorry, I’m not clear on what the solution is with that then.

Maybe I’m doing something wrong here. I am only running my terminal as root user so that I don’t have to constantly be putting my very long password word in for sudo. Is that what you are referring to? Should I run those commands again as my normal login? Is that what you are saying?

Here is the new outputs from my normal log in with your corrected text.

david@lovelace:~$ PID=$(pgrep -u “$USER” nautilus)
pgrep: invalid user name: “david”
david@lovelace:~$ grep Umask /proc/$PID/status
grep: /proc//status: No such file or directory
david@lovelace:~$ eval “$(systemctl --user show -p MainPID dbus)”
“MainPID=1949”: command not found
david@lovelace:~$ grep Umask /proc/$MainPID/status
grep: /proc//status: No such file or directory


david@lovelace:~$ systemctl --user cat dbus | grep UMask=002
UMask=002 # This is the umask i want to use
david@lovelace:~$ 

```