User Tools

Site Tools


castor:stagefiles

Staging files manually

Problem

If you want to run over some rarely used files, it is very likely that they have to be first staged. While running this is done in sequence, so the file is only staged at the moment the algorithm wants to use it. This is of course very time consuming. But there is a possibility to pre-stage the files manually and in parallel.

Solution

Stage the files

Create first a list of the castor file names advanced by -M and store it in variable. So if you have the two files

/castor/cern.ch/grid/lhcb/data/2011/RAW/FULL/LHCb/CALIBRATION11/86753/086753_0000000001.raw
/castor/cern.ch/grid/lhcb/data/2011/RAW/FULL/LHCb/CALIBRATION11/86753/086753_0000000002.raw

and you store them in $FILELIST

echo $FILELIST

should give

-M /castor/cern.ch/grid/lhcb/data/2011/RAW/FULL/LHCb/CALIBRATION11/86753/086753_0000000001.raw -M /castor/cern.ch/grid/lhcb/data/2011/RAW/FULL/LHCb/CALIBRATION11/86753/086753_0000000002.raw

. Afterwards stage them by

stager_get -U <stagejobname> $FILELIST

where <stagejobname> is an arbitrary name use to later control the staging process.

Control staging process

To look if the files are already staged, type

stager_qry -U <stagejobname>

if the staging is still going on the file is marked with STAGEIN, if it is staged it is marked with STAGED.

castor/stagefiles.txt · Last modified: 2012/05/08 08:20 by che