User Tools

Site Tools


castor:transferzh

Copying Files from CASTOR to Zuerich directly

How to

In order to copy files directly from CASTOR to a machine in Zurich, login to the machine grid-ui. There do

lhcb-proxy-init

which will prepare for file-transport. Then do (e.g.):

lcg-cp -v 'srm://srm-lhcb.cern.ch:8443/srm/managerv2?SFN=/castor/cern.ch/user/myInitial/myUsername/myFile' file://home/hep/myUsername/myFile

The -v option just means verbose.

Note: You need a Grid Certificate to do this!

Multi-File Loop

If you want to transfer several files, Create a file containing a list like this

files.txt
/castor/cern.ch/user/n/nchiapol/my-file-1.root
/castor/cern.ch/user/n/nchiapol/my-file-2.root

Now you can use the following one-liner to get all the files

for f in `cat files.txt`; do lcg-cp -v srm://srm-lhcb.cern.ch:8443/srm/managerv2?SFN=$f file://disk/data1/hep/nchiapol/targetDir/${f##*/}; done

${f##*/} is the basename (e.g. my-file-1.root). You could probably use `basename $f` instead.

Transfer with a Script

In /home/hep/decianm/scripts you will find the script castortrans.sh. The best is to make an alias in your .zshrc-file like: alias castortrans=“source /home/hep/decianm/scripts/castortrans.sh” The script can be used the following way:

  • Open a new shell (it's always a good idea to open a new shell for transfers)
  • Type
castortrans -i

This will set up your Grid environment.

  • Type
 castortrans -f yourFileOnCastor -t yourDestinationFile

. If you have the same username at CERN as locally in Zurich, yourFileOnCastor will be the path starting in your home-directory on CASTOR. yourDestinationFile will start in your home-directory in Zurich. If your usernames are not identical in Zurich and at CERN, you have to set them manually in the script.

  • The script has been tested on a z-shell. As it is sourced and not only executed, it can not be guaranteed to work with all the other settings (hence the new shell).

Transfer with an automated Script

If you have several files to download, you can use the script multiplecastortrans.sh in /home/hep/decianm/scripts. The script can be used similarly to castortrans:

  • Open a new shell (it's always a good idea to open a new shell for transfers)
  • Type
multiplecastortrans -i

This will set up your Grid environment.

  • Type
 multiplecastortrans -f yourFileOnCastor -t yourDestinationFile -l lowerBound - u upperBound

With

@COUNTER$

you can add a counter in the path, for example.

multiplecastortrans -f /tuples/123/@COUNTER$/ouputdata/myRootFile.root -t myRootFile@COUNTER$.root -l 2 -u 4

will download the files from the directories

/tuples/123/2/ouputdata/myRootFile.root to myRootFile2.root
/tuples/123/3/ouputdata/myRootFile.root to myRootFile3.root
/tuples/123/4/ouputdata/myRootFile.root to myRootFile4.root
castor/transferzh.txt · Last modified: 2014/02/05 15:26 by nchiap