User Tools

Site Tools


computing:batch

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
computing:batch [2018/09/01 01:52] – [Managing jobs] iwncomputing:batch [2018/09/11 09:51] (current) – [Storage element] iwn
Line 86: Line 86:
   * ''E'' means in error state.   * ''E'' means in error state.
  
-More details on a job can be found with ''qstat -j <//jobid//>''.+More details on a job can be found with ''qstat -j <//jobid//>''. If your jobs are named, you can also use ''qstat -j <//jobname//>'' which might even contain a wildcard '*'.
  ==== Change order of jobs ====  ==== Change order of jobs ====
  
-Change the submission order of the jobs waiting in the queue with ''qalter -js <//jobshare//> <//jobid//>''. The default job share is ''0'' and any integer value (e.g. ''100'') will give the specified job a higher priority. The higher the value, the higher the priority.+Change the order of submission of the jobs waiting in the queue with ''qalter -js <//jobshare//> <//jobid//>''. The default job share is ''0'' and any integer value (e.g. ''100'') will give the specified job a higher priority. The higher the value, the higher the priority.
  
  
Line 106: Line 106:
  ===== Debugging =====  ===== Debugging =====
  
-The T3 TWiki has a page with information on [[https://wiki.chipp.ch/twiki/bin/view/CmsTier3/HowToDebugJobs|debugging jobs interactively]].+ ==== Debugging jobs interactively ====
  
 +The T3 TWiki has a page with information on [[https://wiki.chipp.ch/twiki/bin/view/CmsTier3/HowToDebugJobs|debugging jobs interactively]] with the ''qlogin'' command:
 +<code bash>
 +qlogin -q debug.q -l hostname=t3wn22 -l h_vmem=400M
 +</code>
  ==== Redirecting standard output and error streams ===  ==== Redirecting standard output and error streams ===
 If you want to isolate and save the standard output and the standard error streams (//stdout// and //stderr//) of your main script that would normally prompt in the Terminal window, you can redirect it as usual with ''>>'' and ''2>>'': If you want to isolate and save the standard output and the standard error streams (//stdout// and //stderr//) of your main script that would normally prompt in the Terminal window, you can redirect it as usual with ''>>'' and ''2>>'':
Line 143: Line 147:
  ===== Storage element =====  ===== Storage element =====
  
-You can also copy large output files to the storage element using the copy command **''lcg-cp''** or the recommended **''xrdcp''** from XROOTD, see [[https://wiki.chipp.ch/twiki/bin/view/CmsTier3/HowToAccessSe|How To Acces SE]].+You can also copy large output files to the storage element using the copy command **''lcg-cp''** or the recommended **''xrdcp''** from XROOTD, see [[https://wiki.chipp.ch/twiki/bin/view/CmsTier3/HowToAccessSe|the TWiki on how To Acces SE]] or [[computing:storage|our page on the storage element]].
  
 <code> <code>
Line 167: Line 171:
 gfal-mkdir -p gsiftp://t3se01.psi.ch/`echo $SERESULTDIR | grep -o '/pnfs/psi.ch/.*'` gfal-mkdir -p gsiftp://t3se01.psi.ch/`echo $SERESULTDIR | grep -o '/pnfs/psi.ch/.*'`
 </code> </code>
- 
  ===== Complete example =====  ===== Complete example =====
  
Line 186: Line 189:
  
 If you want to split the events on several jobs you can do it manually as in the example [[computing:batch:splitjobexample|example_splitjobs.py]] and run it with **python**. In this example the command line inputs are maxEvents, firstEvent, inputFileNames and the seed for PU simulation. This works if you first made the CMSSW python script configurable. That can be done following this link [[https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCommandLineParsing|Command line option parsing]]. If you want to split the events on several jobs you can do it manually as in the example [[computing:batch:splitjobexample|example_splitjobs.py]] and run it with **python**. In this example the command line inputs are maxEvents, firstEvent, inputFileNames and the seed for PU simulation. This works if you first made the CMSSW python script configurable. That can be done following this link [[https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideCommandLineParsing|Command line option parsing]].
 +
 +
 + ===== Monitoring busyness on the batch system =====
 +
 +
 +
 +<wrap tip>__Protip__</wrap>:You can see how busy the batch system is due to other users with this command:<code bash>
 +qstat -u \* | tail -n +3 | awk '{if($5=="r"){r[$4]++} j[$4]++} END { for(n in j){ if(r[n]==""){ r[n]=0 } printf "%7s / %-5s - %s\n",r[n],j[n],n }}'
 +</code>
computing/batch.1535759535.txt.gz · Last modified: 2018/09/01 01:52 by iwn