User Tools

Site Tools


castor:transferzh

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
castor:transferzh [2009/03/20 11:11] decianmcastor:transferzh [2014/02/05 15:26] (current) – [How to] nchiap
Line 1: Line 1:
 ====== Copying Files from CASTOR to Zuerich directly ====== ====== Copying Files from CASTOR to Zuerich directly ======
 +** Note: ** [[grid:transferzh | Transfer Instructions for Grid]]
 +
 ===== How to ===== ===== How to =====
-In order to copy files directly from CASTOR to a machine in Zurich, first do+In order to copy files directly from CASTOR to a machine in Zurich, login to the machine grid-ui. There do
 <code> <code>
-export LHCBPRODROOT=/disk/dirac +lhcb-proxy-init
-. ${LHCBPRODROOT}/gLite/external/etc/profile.d/grid-env.sh +
-. ${LHCBPRODROOT}/lcg/etc/profile.d/grid_compat_env.sh +
-export X509_CERT_DIR=/disk/dirac/gLite/external/etc/grid-security/certificates +
-grid-proxy-init+
 </code> </code>
-which will prepare your machine for file-transport.+which will prepare for file-transport.
 Then do (e.g.): Then do (e.g.):
 <code> <code>
Line 17: Line 15:
  
 ** Note: ** You need a Grid Certificate to do this! ** Note: ** You need a Grid Certificate to do this!
 +
 +
 +===== Multi-File Loop =====
 +If you want to transfer several files, Create a file containing a list like this
 +<file txt files.txt>
 +/castor/cern.ch/user/n/nchiapol/my-file-1.root
 +/castor/cern.ch/user/n/nchiapol/my-file-2.root
 +</file>
 +
 +Now you can use the following one-liner to get all the files
 +<code bash>
 +for f in `cat files.txt`; do lcg-cp -v srm://srm-lhcb.cern.ch:8443/srm/managerv2?SFN=$f file://disk/data1/hep/nchiapol/targetDir/${f##*/}; done
 +</code>
 +''${f##*/}'' is the basename (e.g. my-file-1.root). You could probably use ''`basename $f`'' instead.
  
 ===== Transfer with a Script ===== ===== Transfer with a Script =====
 In ''/home/hep/decianm/scripts'' you will find the script ''castortrans.sh''. The best is to make an alias in your ''.zshrc''-file like: ''alias castortrans="source /home/hep/decianm/scripts/castortrans.sh"'' In ''/home/hep/decianm/scripts'' you will find the script ''castortrans.sh''. The best is to make an alias in your ''.zshrc''-file like: ''alias castortrans="source /home/hep/decianm/scripts/castortrans.sh"''
 +The script can be used the following way:
 +  * Open a new shell (it's always a good idea to open a new shell for transfers)
 +  * Type 
 +<code>
 +castortrans -i
 +</code>
 +This will set up your Grid environment.
 +  * Type 
 +<code> castortrans -f yourFileOnCastor -t yourDestinationFile
 +</code>
 +. If you have the same username at CERN as locally in Zurich, yourFileOnCastor will be the path starting in your home-directory on CASTOR. yourDestinationFile will start in your home-directory in Zurich. If your usernames are not identical in Zurich and at CERN, you have to set them manually in the script.
 +  * The script has been tested on a z-shell. As it is sourced and not only executed, it can not be guaranteed to work with all the other settings (hence the new shell).
  
 +===== Transfer with an automated Script =====
 +If you have several files to download, you can use the script ''multiplecastortrans.sh'' in ''/home/hep/decianm/scripts''.
 +The script can be used similarly to castortrans:
 +  * Open a new shell (it's always a good idea to open a new shell for transfers)
 +  * Type 
 +<code>
 +multiplecastortrans -i
 +</code>
 +This will set up your Grid environment.
 +  * Type 
 +<code> multiplecastortrans -f yourFileOnCastor -t yourDestinationFile -l lowerBound - u upperBound
 +</code>
 +With <code>
 +@COUNTER$
 +</code>
 +you can add a counter in the path, for example.
 +<code>
 +multiplecastortrans -f /tuples/123/@COUNTER$/ouputdata/myRootFile.root -t myRootFile@COUNTER$.root -l 2 -u 4
 +</code>
 +will download the files from the directories 
 +<code>
 +/tuples/123/2/ouputdata/myRootFile.root to myRootFile2.root
 +/tuples/123/3/ouputdata/myRootFile.root to myRootFile3.root
 +/tuples/123/4/ouputdata/myRootFile.root to myRootFile4.root
 +</code>
  
- ====== 
  
castor/transferzh.1237543879.txt.gz · Last modified: 2009/03/20 11:11 by decianm