File Management with SRM and DPM

DPM-client

These set of client tools allow direct interaction with DPM-type Storage Elements, the ones deployed in our grid infrastructure.

They should not be generally used.

To use DPM client tools, you should set the DPNS_HOST env variable with the hostname of the Storage Element you want to interact to:

[morgan@localhost ~]$  ldapsearch -x -LLL -h egee-bdii.cnaf.infn.it:2170 -b "o=grid" '(&(GlueSEImplementationName=DPM)(GlueSEName~=CNR))' GlueSEUniqueID
dn: GlueSEUniqueID=dpm.research-infrastructures.eu,Mds-Vo-name=CNR-PROD-PISA,Mds-Vo-name=local,o=grid
GlueSEUniqueID: dpm.research-infrastructures.eu

dn: GlueSEUniqueID=gridse.ilc.cnr.it,Mds-Vo-name=CNR-ILC-PISA,Mds-Vo-name=local,o=grid
GlueSEUniqueID: gridse.ilc.cnr.it

[morgan@localhost ~]$ ldapsearch -x -LLL -h egee-bdii.cnaf.infn.it:2170 -b "o=grid" '(&(GlueChunkKey=GlueSEUniqueID=gridse.ilc.cnr.it)(GlueSAAccessControlBaseRule=VO:planck))' GlueSAPath
dn: GlueSALocalID=planck,GlueSEUniqueID=gridse.ilc.cnr.it,Mds-Vo-name=CNR-ILC-PISA,Mds-Vo-name=local,o=grid
GlueSAPath: /dpm/ilc.cnr.it/home/planck

[morgan@localhost ~]$ export DPNS_HOST=gridse.ilc.cnr.it

Some of the DPM client commands:

• dpns-ls - Used to list/browse directories into the set Storage Element

• dpns-mkdir - Used to create a new subdirectory

• dpns-chmod/dpns-chown - Used to change owner and permissions to files

• dpns-setacl/dpns-getacl - Used to get/set ACLs of SEs' files

All the previous commands interact with the SEs machines using the SRM interface.

Dig into LFC and DPM relations

Using lcg-cr command we can upload a file as shown in the LFC tutorial

Let's have a look to what happened:

[morgan@localhost ~]$ lfc-ls /grid/planck/morgan
[morgan@localhost ~]$ lcg-cr --vo planck -d $DPNS_HOST -l lfn:///grid/planck/morgan/test.txt file:/home/morgan/delme.txt 
guid:94f656b7-e030-41e4-ae43-99c99c6a7f10
[morgan@localhost ~]$ lcg-lr guid:94f656b7-e030-41e4-ae43-99c99c6a7f10
srm://gridse.ilc.cnr.it/dpm/ilc.cnr.it/home/planck/generated/2008-08-20/file914135ea-cd64-407e-80dc-414014b4ba3d
[morgan@localhost ~]$ dpns-ls -l dpm/ilc.cnr.it/home/planck/generated/2008-08-20/file914135ea-cd64-407e-80dc-414014b4ba3d
dpm/ilc.cnr.it/home/planck/generated/2008-08-20/file914135ea-cd64-407e-80dc-414014b4ba3d: invalid path
[morgan@localhost ~]$ dpns-ls -l /dpm/ilc.cnr.it/home/planck/generated/2008-08-20/file914135ea-cd64-407e-80dc-414014b4ba3d
-rw-rw-r--   1 118      111                       6 Aug 20 15:30 /dpm/ilc.cnr.it/home/planck/generated/2008-08-20/file914135ea-cd64-407e-80dc-414014b4ba3d

So, as expected, the lcg-cr command both upload the file into the SE and register its SURL (Site URL) into the File Catalog.

Using globus-url-copy and lcg-rf

We create a directory on a DPM SE and copy a file there, then we register the file on the catalogue.

[morgan@localhost ~]$ dpns-ls -l /dpm/ilc.cnr.it/home/planck/
drwxrwxr-x 101 root     111                       0 Aug 01 05:37 generated
[morgan@localhost ~]$ dpns-mkdir  /dpm/ilc.cnr.it/home/planck/Simulations
[morgan@localhost ~]$ dpns-ls -l /dpm/ilc.cnr.it/home/planck/
drwxrwxr-x   0 118      111                       0 Aug 20 16:09 Simulations
drwxrwxr-x 101 root     111                       0 Aug 01 05:37 generated
[morgan@localhost ~]$ globus-url-copy file:/home/morgan/delme.txt gsiftp://gridse.ilc.cnr.it/dpm/ilc.cnr.it/home/planck/Simulations/input.0
[morgan@localhost ~]$ dpns-ls -l /dpm/ilc.cnr.it/home/planck/Simulations
-rw-rw-r--   1 118      111                       6 Aug 20 16:14 input.0
[morgan@localhost ~]$ lcg-r
lcg-ra               lcg-rec-cp           lcg-rec-cr           lcg-rec-del          lcg-rep              lcg-replica-manager  lcg-rf
[morgan@localhost ~]$ lcg-rf --vo planck -l lfn:/grid/planck/morgan/input.0 srm://gridse.ilc.cnr.it/dpm/ilc.cnr.it/home/planck/Simulations/input.0
guid:8a6d677b-26cb-4880-9516-334dcbffc9a2
[morgan@localhost ~]$ lfc-ls -l /grid/planck/morgan/
-rw-rw-r--   1 101      102                       6 Aug 20 16:18 input.0
-rw-rw-r--   1 101      102                       6 Aug 20 15:29 test.txt

Actually lcg-cr is using the steps done in this paragraph (the only difference is that lcg-cr will upload the file into a directory named as the current date and generate and random and unique file name)

Understanding Grid File Permissions

Let's have a look to the permission of a entry in the File Catalogue:

[morgan@localhost ~]$ lfc-ls -l /grid/planck/morgan/
-rw-rw-r--   1 101      102                       6 Aug 20 16:18 input.0
-rw-rw-r--   1 101      102                       6 Aug 20 15:29 test.txt
[morgan@localhost ~]$ lfc-getacl /grid/planck/morgan/input.0
# file: /grid/planck/morgan/input.0
# owner: /C=IT/O=INFN/OU=Personal Certificate/L=INAF Trieste/CN=Giuliano Taffoni
# group: planck
user::rw-
group::rw-              #effective:rw-
other::r--

So both the owned and the planck group will have full permission on the entry.

Let's change the permissions of the file denying the access to the member of the planck group, using lfc-chmod.

[morgan@localhost ~]$ lfc-chmod 600 /grid/planck/morgan/input.0
[morgan@localhost ~]$ lfc-getacl /grid/planck/morgan/input.0
# file: /grid/planck/morgan/input.0
# owner: /C=IT/O=INFN/OU=Personal Certificate/L=INAF Trieste/CN=Giuliano Taffoni
# group: planck
user::rw-
group::---              #effective:---
other::---

So, only the owner should access the file. However this file can be downloaded ANYWAY! (you can check!). In fact, if you check on the SE

[morgan@localhost ~]$ dpns-ls -l /dpm/ilc.cnr.it/home/planck/Simulations
-rw-rw-r--   1 118      111                       6 Aug 20 16:14 input.0
[morgan@localhost ~]$ dpns-getacl /dpm/ilc.cnr.it/home/planck/Simulations/input.0
# file: /dpm/ilc.cnr.it/home/planck/Simulations/input.0
# owner: /C=IT/O=INFN/OU=Personal Certificate/L=INAF Trieste/CN=Giuliano Taffoni
# group: planck
user::rw-
group::rw-              #effective:rw-
other::r--

The file on the Storage Element HAS still group permissions allowing gilda user full access. So this is why, the lfc-chmod command will change permission ONLY on the File Catalogue's entry and NOT on the actual SE replicas.

So if you want to deny access to gilda group members, the owner should set permission manually also for the SE's file:

[morgan@localhost ~]$ dpns-chmod 600 /dpm/ilc.cnr.it/home/planck/Simulations/input.0
[morgan@localhost ~]$ dpns-getacl /dpm/ilc.cnr.it/home/planck/Simulations/input.0
# file: /dpm/ilc.cnr.it/home/planck/Simulations/input.0
# owner: /C=IT/O=INFN/OU=Personal Certificate/L=INAF Trieste/CN=Giuliano Taffoni
# group: planck
user::rw-
group::---              #effective:---
other::---

Now the file cannot be downloaded.:

[morgan@localhost ~]$ lcg-cp -v --vo planck  lfn:/grid/planck/morgan/input.0 file:$HOME/test
lcg_cp: Permission denied

The Conclusion: IMPORTANT: File Permissions and ACLs are not in sync between the File Catalogue and the Storage Elements! This should be done manually by the user!

-- TaffoniGiuliano - 19 Aug 2008