Revision as of 20:28, 1 March 2018

This guide will show you how to manage sharing permissions on the cluster for 4 common scenarios, and explain how things work along the way to help you work effectively with other users and groups.

Let's say:

1. You have a directory called you10001/ on the scratch filesystem.
2. Your username is you10001.
3. You want to give your lab mate thy20002 access to a project in the directory partner/ which you are working on together.
4. Your professor / advisor's name is adv90009.
5. You don't want anyone to see the files in private/ except allow your lab mate and advisor to read your project notes notes_project_partner.txt.
6. You want all of your lab mates to have access to lab/
7. Your directories follow this structure:

[you10001@cn02 ~]$ tree -d /scratch/you10001
/scratch/you10001
├── collaborator
├── lab
├── partner
└── private

4 directories

It just so happens that the alphabetical order of the directories in the list above also matches how restrictive we want the directories to be, with collaborator/ being the least restrictive and private/ being the most restrictive.

Let's get started!

Getting started

You can only share or restrict directories that you own. If someone else owns a directory, say thy20002 owned the directory, you should ask thy20002 to come and read this page to help manage the sharing, or if thy20002 is no longer working at the lab, you should e-mail the cluster admins to change ownership over to you (which they will do using the chown command) and cc your advisor so that the admins can confirm with your advisor.

You can check whether you own a directory among many other things using the getfacl command:

[you10001@cn02 ~]$ alias getfacl='getfacl -p'

[you10001@cn02 ~]$ getfacl /scratch/you10001/
# file: scratch/you10001/
# owner: you10001
# group: Domain_Users
user::rwx
group::r-x
other::r-x

You can see you are the owner! These permissions are the default for a directory you create. We will see how to customize these settings as we go through this guide. By the way, we used the aliascommand to avoid the annoying message we would otherwise get of "getfacl: Removing leading '/' from absolute path names".

There are two ways to share files and directories:

1. The normal UNIX permissions and managing those permissions with the chmod and chown commands, and
2. The more modern and powerful ACL (Access Control List) system managed with the setfacl command.

Case 1: private directory

To make a directory private to yourself, remove both the "group" and "other" permissions. This is what our private/ directory looks like with the default permissions:

[you10001@cn02 ~]$ getfacl /scratch/you10001/private/
# file: scratch/you10001/private/
# owner: you10001
# group: Domain_Users
user::rwx
group::r-x
other::r-x

Remove the read permissions from private/ using chmod, and check the changes using getfacl:

[you10001@cn02 ~]$ chmod g-rx,o-rx /scratch/you10001/private/

[you10001@cn02 ~]$ getfacl /scratch/you10001/private/
# file: scratch/you10001/private/
# owner: you10001
# group: Domain_Users
user::rwx
group::---
other::---

Now if someone else were to try and see the private/ directory they would see "Permission denied":

[thy20002@cn02 ~]$ ls /scratch/you10001/private/
ls: cannot open directory /scratch/you10001/private/: Permission denied

It's important to understand 2 things about making files and directories private on our cluster:

1. The Domain_Users group is special to our cluster, and that group includes all users. This means one might want to treat the "group" Domain_Users as no different that "other". This universal Domain_Users group is an idiosyncrasy of our cluster that's not typical of most UNIX filesystems.
2. Files can be private even if they have read permission enabled, if there is any directory in the path that blocks those read permissions. For example everything in your home directory is private to you because your you10001 directory is private. This is why even though your ~/.bashrc has read permissions for everyone it is actually private:

[thy20002@cn02 ~]$ ls /home/you10001/
ls: cannot open directory /home/you10001/: Permission denied

[you10001@cn02 ~]$ namei -l ~/.bashrc
f: /home/pan14001/.bashrc
dr-xr-xr-x root     root         /
drwxr-xr-x root     root         home
drwx------ you10001 Domain_Users you10001
-rw-r--r-- you10001 Domain_Users .bashrc

[you10001@cn02 ~]$ getfacl /home/you10001/
# file: /home/you10001/
# owner: you10001
# group: Domain_Users
user::rwx
group::---
other::---

Sometimes the admins are asked "can you share this file in my home directory with my lab mate"? This is a reasonable question, but is technically difficult. #Exceptions inside a private directory explains why this is not a good arrangement. The recommended way to share is to move your the file or directory somewhere else outside of the purely private directory.

Case 2: partner project

Case 3: multiple users in a lab

Case 4: collaborators outside of the lab

Using ACLs: more flexible

Normal UNIX permissions managed through chown, and chmod, are extremely limited. One major limitation is that they do not allow you to provide access to just one other user, you have to provide access to a group of users, or all users. Since non-adminstrators cannot create UNIX groups, you cannot pass a file off one/more users without asking the HPC admin team to make you a group. File access control lists (facl's) solve this problem by allowing you to allow specific users/groups to access your files. The primary advantage to this is that you do not need to contact our support team to provide shared work space, and can instead create shared directories yourself.

To give one other user some permissions(same format as UNIX permissions) to access your file, you can do something akin to the following:

$ setfacl -m "u:user_name:permissions' example_file

To make your permission permeate to all new subdirectories and files you can set a default ACL for some directory.

$ setfacl -dm "u:user_name:permissions' example_dir

Properly set default ACLs, will be functionally equivalent to the shared work directories we can create for you. The advantage again being that you can make these shared work directories yourselves, with an added advantage that you can make any file or directory readable by just your colleagues. You can even make your home directory accessible.

To create a shared directory on our temporary file system you would do:

$ mkdir /scratch/shared_dir_name
$ setfacl  -m  "u:your_user_name:rwx" -m   "u:associate_1_user_name:rwx" -m   "u:associate_2_user_name:rwx" /scratch/shared_dir_name
$ setfacl -m "d:u:your_user_name:rwx" -m "d:u:associate_1_user_name:rwx" -m "d:u:associate_2_user_name:rwx" /scratch/shared_dir_name

You can add any number of "u:user_name:perm" fields to the above command while replacing the italicized areas appropriately.

To check some file or directories Access Control List you can do:

$ getfacl dir_or_file

For more information you can go to:

https://wiki.archlinux.org/index.php/Access_Control_Lists
http://vanemery.net/Linux/ACL/linux-acl.html#scenario

Using native permissions: more

If you're in a large group, or your work requires persistent storage, you should open a ticket with Storrs HPC support, and we will evaluate if it necessary to provide disk space on /shared/, or /archive/.

Appendix

Exceptions inside a private directory

Making exceptions in a purely private directory is hard but not impossible. In general it's a bad idea and makes life complicated as we will see in the rest of this sub-section, and instead the better thing to do when you want to share a file is to move it somewhere else outside of the purely private directory.

But for the sake of better learning about how the permission system works, let's try making an exception. In our case we want our lab mate thy20002 to be able to see our notes_project_partner.txt document inside the private/ directory. Give read permission to thy20002 only using setfacl:

[you10001@cn02 ~]$ setfacl -m u:thy20002:r /scratch/you10001/private/notes_project_partner.txt

[you10001@cn02 ~]$ getfacl /scratch/you10001/private/notes_project_partner.txt 
# file: /scratch/you10001/private/notes_project_partner.txt
# owner: you10001
# group: Domain_Users
user::rw-
user:thy20002:r--
group::r--
mask::r--
other::r--

However still our lab mate cannot see the file:

[thy20002@cn02 ~]$ ls /scratch/you10001/private/notes_project_partner.txt
ls: cannot access /scratch/you10001/private/notes_project_partner.txt: Permission denied

This is because the entire chain of directories needs to have read permission, but we took it away with the chmod command. The namei command helps show why our lab mate cannot see the file by inspecting all the directories in the path:

[you10001@cn02 ~]$ namei -l /scratch/you10001/private/notes_project_partner.txt 
f: /scratch/you10001/private/notes_project_partner.txt
dr-xr-xr-x root     root         /
lrwxrwxrwx root     root         scratch -> /gpfs/scratchfs1
dr-xr-xr-x root     root           /
drwxr-xr-x root     root           gpfs
drwxrwxrwt root     root           scratchfs1
drwxr-xr-x you10001 Domain_Users you10001
drwx------ you10001 Domain_Users private
-rw-r--r-- you10001 Domain_Users notes_project_partner.txt

So we need to add an exception to the chmod command.

[you10001@cn02 ~]$ setfacl -m u:hpc-pan:rx /scratch/you10001/private/