Permissions issues on local external folders

ℹ️ Support
1

Sorry to be making another support topic but I though tit was important to keep this separate from other issues threads I had made for the sake of clarity.

My setup:

Nextcloud version: 13
Operating system and version: Ubuntu 18.04 and Windows 10
Apache
PHP version: 7.1 I believe

image
In summary:

  • I have a windows laptop.
  • I have an Ubuntu desktop.
  • The desktop is running the server.
  • The desktop has two local external folders mounted to the NC server.
    1. The Archive which is a 3TB drive used for long term storage and large media files.
    2. my work folder which is physically located at /home/david/work in my user home folder.
  • The work folder is synced to my laptop using the NextCloud client.
  • The Archive is mounted as a webdav network folder giving me access to the 3TBs of files if I need to get something off my harddrive and back into the archive, or in case I need to pull a file out of the archive onto my Laptop to work with.

My Problem

When I edit a file in my work folder on my laptop (which is located at C:\Users\david\work) the changes will be shot up to NC through the sync client. Those changes will be then be made to the file by NC under the user www-data to the twin file at /home/david/work.

When it does this, I find a file in my work folder that I, as a user, cannot edit, delete, rename, or move, because the owner is www-data. I can use sudo or change the file’s owner, but if I add another file to the folder on the laptop, it will have the same problem, so it’s not a very good longterm solution.

The opposite is also true. If I directly edit a file locally in /home/david/work, NextCloud has trouble adding it to it’s database because it isn’t owner of it. I get a bunch of sync errors from both sides, saying things like “cannot access that file or directory” or “you lack the necessary permissions to upload or create files here”

I tried adding myself to the usergroup www-data, and I tried adding www-data to the usergroup david, but by default it seems that only the owner of www-data has the ability to edit or delete files and putting myself or www-data into either of those categories does not work.

I have already checked Documentation and the forums.

But I can not seem to find an answer.

How can I fix this?

2

This is the important part for fab’s fix, but I’m pretty sure it doesn’t apply to your use-case. These are NTFS-specific options to mount his volume with the www-data user and group as owners. The umask 002 part allows group members to modify the files, but that’s not something you want to specify on your whole / (or even /home) volume.

It would seem that what you want is for group-members to be able to write to these files/folders. For a one-off solution,chmod --recursive g+rw /home/david/work should allow the www-data user (since it’s a member of your ‘group’) to do this, assuming Apache and AppArmor aren’t involved.
As an ongoing thing, new files or folders created within this directory will have different ownership and different permissions. I can suggest two options:

  • Hacky solution: also run chown/chmod when you run your files:scan, so these things are corrected quickly
  • Proper solution: Take a dive and look into how you can make sure these new files get created with the correct permissions. You might want to start with “Understanding umask Value Usage”, but I’ve never gone deep enough to give you the gist. Perhaps someone else has?
3

Okay yeah the umask thing sounds like what I need to do.

I typed umask 003 in command line and then I mkdir a test directory. I checked it’s permissions and they look like what I intended.

But if I create a new folder inside of Nautilus etc, it does not have the correct permissions. I am guessing that the umask for a user in CLI is different than when they are using Nautilus. In the case of Nautlius, the user is actually nautilus not david? And that means that in order to change the umask I need to change it for www-data.

Also having read that stuff I now understand your first statement about 002. Why wouldn’t I want to give umask to my work folder on my home volume? It would only open it up to attack from people within the groups, so www-data and david, and I’m pretty confident about those being safe… right? Why shouldn’t I make it so the folders are mounted with the www-data user and groups?

4

Pretty much your options as far as umask goes are:

  1. Suggest a umask for all programs acting on behalf of a given user (or two users, in your case), but the programs have the option to override it.
  2. Enforce the umask on an entire given volume, because the filesystem of that volume doesn’t support Unix permissions

Nautilus is acting on your behalf and running as your user account (david), but the umask permissions are only a suggestion/default, which Nautilus may choose to ignore. I’d also make sure you’ve entirely logged out and back in, in case Nautilus spawned from a process running before you set your umask, and didn’t think to check. If Nautilus continues to ignore the umask after that, then the site I linked suggests cleaning up with chmod as needed.
That said, yes, you would also need to set the umask for the www-data user so that (hopefully) Nextcloud would follow that default/suggestion of permissions for anything it created. If Nextcloud and Nautilus obey your umask suggestion, this should work smoothly, and you won’t have to worry about the stuff I mention next.

You can’t only set it for a given folder within a volume - it would have to be set for the whole volume, to my knowledge.
The command you used gives all programs running as david a suggested set of permissions to use when creating files. That’s option 1.

Option 2: The mount option they used gives the Linux kernel a set of permissions it should pretend the NTFS volume in question has, since NTFS doesn’t support Unix-style permissions. If you want to create a new partition of a type that doesn’t support Unix permissions (any such filesystem is listed listed with a umask option in man mount), then mount it as your /home/david/work folder, that could work. I think you’ll likely get more mileage out of using chmod on anything that appears with the wrong permissions, or finding a file manager that respects your umask settings (assuming Nautilus is the only thing ignoring your umask command, and continues to ignore it after a logout or reboot).

5

So doing some more research I’ve come to similar conclusions as you. I need to change the usmask for www-data. From my reading it seems that I can change my own umask fairly easily by changing my .bashrc or .profile to include a command of umask 003 The problem is… I can’t find out how to change the umask for www-data. All the pages I find talk about it adding the umask command to the startup scripts etc for apache2 or checking under /home/www-data etc, but none of the those files that they suggest changing actually exist. I think I understand that what each of them is suggesting is to change the equivlant of the .bashrc for www-data but again, that doesn’t do me much good if I can’t find the .profile file or envars for apache2 etc.

6

Found a couple of ServerFault questions, in the hopes it helps.


This one’s got a ton of things the OP tried, which might actually be helpful. Make sure you do the full service apache2 stop then start, rather than just a reload when you try these out.

The first answer from this really similar question includes a way to test to make sure you’ve applied it correctly to the Apache server process. That could be handy in finding out if Nextcloud continues to ignore your umask even after it’s applied properly to Apache.

7

Yeah that’s the thread I was looking at but I don’t have an envars folder in apache etc. Also I’m running nginx anyway.

but I did find /etc/profile/nginx once I actually went to the command line. I couldn’t see profile even with hidden folders set to visible for some reason in nautilus, but once I tried in CLI I found it. I added umask there. and I’ve stopped and started the server and I’m fixing to check and see if it worked.

As for the script for checking if stuff works… uh… I actually don’t know how to run that script?EDIT…Figured out how to do the php script. Save it as a file and then run it using php- f script.php

Anyway I’m having to shift some stuff around because my folders got kind of messy in the process of trying to back stuff up. I don’t think I have the group shared permissions yet the way I want, but I’m going to wait till after this copy and sync process finishes.

I also found out how to remove the file types limitations that I was having trouble with. I edit the file sync-excluded.lst which was under the applications folder for the NextCloud Client, and on Windows was in the Program Files for NextCloud.

I removed a bunch of limitations for temp files and stuff like *_.* type stuff.

8

Hey @mactrent So I fiddled around with the umask thing, and I may have got it working, or may have not, but I think I’m running into a problem on the opposite end of the equation.

When I create a file on my Windows laptop, it can’t sync it up to the cloud, saying that it doesn’t have permission to create subfolders. I don’t know what that means though. But from what I can tell looking up that specific wording it seems to be an issue with the client, not an issue with the permissions set on the local external folders of my desktop server. If it were just that then the client should be able to at least push the folders up to the web GUI etc.

9

Hmm, and are you able to create files from the web GUI then? If there is (or was) an actual permissions issue on the server side, the web GUI will usually show a warning “you can’t create files and folders here” or similar. If you fixed the permissions recently, you might need to do an occ files:scan before Nextcloud realizes it’s fixed.

10

So I recreated all the folders from scratch and I think I figured out that issue though not sure how to fix it.

I can create a folder in Work on my desktop and it will be visible in the web interface and will sync down onto the laptop. But I can’t add any folders to that folder from the web interface, and if I put a folder in it on my laptop, the sync client will refuse to let it through saying I don’t have permission.

If I create a folder on web GUI, it will appear on my desktop folder, but it will have a little lock symbol on it. It’s owner will be www-data.

If I create a folder on my laptop it will sync up to the web GUI via the client, and then appear on my desktop but also be impossible for me to edit and the owner will be www-data.

My Resources folders etc are able to be accessed etc by webdav because I had previously chmod 775 them. But I imagine I would have the same problems that I’m having with the Work folder if I added folders to them through the Web GUI etc.

I can’t chmod things every single time I create a folder, obviously, so this really just brings us back to the original problem of umask etc. All these different variations of behavior are based on the default way that www-data grants permissions on new files it creates. So I need to fix that. I will try again with the instructions you linked and see if now that I have a better understanding of the process, if I can’t perhaps fix this. I just need to be more systematic.

11

Hey I think I got everything fixed. I’m fixing to do a reboot to double check but I wanted to log some quick notes here for anyone else who might have this or a similar problem in the future.

In order to do this I first added david to the www-data usegroup and added www-data to the david usergroup.
This might be a little bit of a security issue but I really don’t think it’s that major.

Secondly I needed to change the umask for www-data, so that when it created a new file in the folder it would do so with a 775 permission (meaning that user and groups would have full write, read, execute access).

This can be done on my nginx server by doing the thing describerd here: ubuntu - Nginx/php-fpm umask setting - Stack Overflow

if you use systemd [i.e. Ubuntu 16.04], then edit /lib/systemd/system/php5-fpm.service
And edit chapter “Service”:

[Service]
UMask=0002

In my case I edited php-fpm 7.2 because that’s the version I’m using.

Okay so now www-data will create new folders on the system that can be opened by the groups etc.

But the next thing to do is to make it so that files made by the user are editable to the www-data. The way to do that is to change the umask for david. It’s enough to just add the line umask 002 somewhere in your .bashrc or .profile file. If you want to you can also add it to your root user account in a similar way.

This still isnt’ enough though because as mactrent said, nautilus etc are going to have their own default umasks to use when you create folders etc with them. According to this post it is possible like this: gnome3 - How to set `umask` for the entire gnome session? - Unix & Linux Stack Exchange

up vote 2 down vote

The problem is that mentioned by Sebasth. I tried many things, but then i found a workaround which consists in overwriting the (per-user) UMask of dbus:

$ systemctl --user edit dbus

In the file that gets opened, just write:

[Service]
UMask=002 # This is the umask i want to use

The file gets saved in .config/systemd/user/dbus.service.d/override.conf and overrides the dbus default umask, which i presume is inherited from systemd --user, since dbus is launched by it. Just logout and login again and gnome applications should use the specified umask. It works form me.

I’m going to restart my machine and see if it works but I think it should.

1 Like
12

Bad news, it does not seem that the changes meant to force nautlius to recognize the umask 0002 are working. I also tried going into /etc/login.defs which has some kind of setting for changing group umask defaults, though the wording of the instructions is a little hard to parse.

I’ve tried changing multiple different files and nothing seems to work.

Is there a way that I can make nextcloud write as the user david instead of www-data?

16

Okay so I might not be understanding the technology very well but it sounds like to use the bindfs technique I have to use the Fuse filesystem which means I can’t use my current NTFS (for my archive drive) or Ubuntu Logical Volume (for my SSD that the OS runs on) ?

To do that solution I’d basically need to reinstall linux etc right?

Not having the ability to copy in files to the directory with correct permissions would not be ideal, but I’m also so far along in this process I don’t know if I’m really up to restart everything and use a different filesystem that would be unreadable to my Windows partition. Maybe I’m misunderstanding the ideas though.

root@lovelace:~# systemctl --user cat dbus | grep UMask=002
Failed to connect to bus: No such file or directory

I can confirm that each time I’ve tried this stuff etc I’ve completely rebooted to make sure it restarted the stuff correctly.

Further results requested:

root@lovelace:~# eval “$(systemctl --user show -p MainPID dbus)”
Failed to connect to bus: No such file or directory
“”: command not found
root@lovelace:~# grep Umask /proc/$MainPID/status
grep: /proc//status: No such file or directory

and

root@lovelace:~# PID=pgrep -u “$USER” nautilus
-u: command not found
root@lovelace:~# grep Umask /proc/$PID/status
grep: /proc//status: No such file or directory
20

Okay so would I then basically just turn my Work directory into a bindfs filesystem? Sorry, I’m not clear on what the solution is with that then.

Maybe I’m doing something wrong here. I am only running my terminal as root user so that I don’t have to constantly be putting my very long password word in for sudo. Is that what you are referring to? Should I run those commands again as my normal login? Is that what you are saying?

Here is the new outputs from my normal log in with your corrected text.

david@lovelace:~$ PID=$(pgrep -u “$USER” nautilus)
pgrep: invalid user name: “david”
david@lovelace:~$ grep Umask /proc/$PID/status
grep: /proc//status: No such file or directory
david@lovelace:~$ eval “$(systemctl --user show -p MainPID dbus)”
“MainPID=1949”: command not found
david@lovelace:~$ grep Umask /proc/$MainPID/status
grep: /proc//status: No such file or directory


david@lovelace:~$ systemctl --user cat dbus | grep UMask=002
UMask=002 # This is the umask i want to use
david@lovelace:~$ 

```
22

you mean I should use the command:
PID=$(pgrep -u '$USER' nautilus) instead of PID=$(pgrep -u “$USER” nautilus)?

24

Oh yes! gotcha.

david@lovelace:~$ eval "$(systemctl --user show -p MainPID dbus)"
david@lovelace:~$ PID=$(pgrep -u "$USER" nautilus)
david@lovelace:~$

It doesn’t seem to do anything.

Oh wait I also need to run the second commands.

Here’s their output:

david@lovelace:~$ grep Umask /proc/$MainPID/status
Umask:	0022
david@lovelace:~$ eval “$(systemctl --user show -p MainPID dbus)”
“MainPID=1949”: command not found
david@lovelace:~$
26

david@lovelace:~$ grep Umask /proc/$MainPID/status
Umask: 0022
david@lovelace:~$ grep Umask /proc/$PID/status
Umask: 0022

That bindfs thing sounds like the best solution though. let me look at your link more and see how to do it and stuff.

28

Yeah I realized the curly quotes thing again so I checked back. Both are coming back 0022.

And heres’ the output you requested:
david@lovelace:~$ systemctl --user cat dbus | grep UMask=002
UMask=002 # This is the umask i want to use

29

With bindfs the command I could use would be:

 bindfs --mirror-only=www-data l ~/Work

Correct? And then I put it in fstab so it does that everytime it mounts the drives etc at start up.

31

I’m not quite clear what to do with those commands. But entered them in. When I tried killall strace it told me that there was no process.

Here’s a sampler of when I run the syscall.log. It was the only command that I ran that seemed to give me any meaningful output:

david@lovelace:~$ strace -p 1 -f >"syscall.log" 2>&1 &
[1] 1412
david@lovelace:~$ strace -p 1 -f >"syscall.log" 2>&1 &
[2] 1448
[1]   Exit 1                  strace -p 1 -f > "syscall.log" 2>&1
david@lovelace:~$ strace -p 1 -f >"syscall.log" 2>&1 &
[3] 1450
[2]   Exit 1                  strace -p 1 -f > "syscall.log" 2>&1

As stuff is acting wonky, do you think I should just instead try the bindfs solution etc? Is my bindfs command I linked earlier correct syntax? I read the page and that seems to be what I want to do, though when I ran it, I got an error.

BUT!!! It looks like now the umask is set correctly! When I ran systemd restart that must have restarted it correctly. I rebooted my computer a couple of times but hadn’t in the last ocuple of hours and maybe in the process I fixed my dbus setting to be more correct.

david@lovelace:~$ PID=$(pgrep -u “$USER” nautilus)
david@lovelace:~$ grep Umask /proc/$PID/status
Umask: 0002

Now, the question I guess I have is… do I need to do the bindfs stuff anyway because it would allow me to move files from other folders etc?