Differences

This shows you the differences between two versions of the page.

--- ganga:ganga [2011/09/11 16:32] – [Submitting a Job on the local batch system in Zurich (PBS)] decianm
+++ ganga:ganga [2017/01/12 12:23] – bernet
@@ Line 5: / Line 5: @@
 ===== Where to run =====
 Ganga can be run on every lxplus node. The preferred option however is to run it locally from Zurich on grid-ui, as there are no (less) disk quota constraints.
+===== Ganga configuration on Zurich Grid Cluster =====
+Before starting Ganga the first time on the Zurich cluster, download and install the following .gangarc file in your home directory. This will create the correct Ganga environement for you.
 ===== Starting Ganga on lxplus / grid-ui =====
@@ Line 84: / Line 88: @@
 ===== How to resubmit all failed subjobs =====
 Sometimes it is also useful to ban a site and resubmit the jobs.
+<code python>
+for js in jobs(24).subjobs.select(status='failed').ids():
+   js.backend.settings['BannedSites'] = ['LCG.CERN.ch'] #optional
+   js.resubmit()
+</code>
+or
 <code python>
 for job in jobs.select(5,11):
@@ Line 393: / Line 403: @@
 which will take 20 files per job.
+===== Submitting a ROOT-job =====
+It can be useful to submit ROOT-jobs to the Grid via Dirac. You can submit ''.C''-ROOT-scripts or python scripts. If your script needs external libraries, they can be added to the inputsandox like
+<code python>
+j = Job()
+j.name = 'my great ROOT job'
+j.inputsandbox = [ File ( name = '/somePath/lib1.so'), File ( name = '/somePath2/lib2.so')]
+</code>
+You can also pass arguments to a script. Suppose you have a script ''myScript.C'' like:
+<code C>
+void myScript(int a, int b){
+     std::cout << "a: " << a " b: " << b << std::endl;
+     // -- Do something useful
+     }
+</code>
+and you would like to pass intput arguments for a and b. This can be done with the following piece of code:
+<code python>
+j.application = Root (
+    version = '5.34.00' ,
+    usepython = False ,
+    args = [1,2],
+    script = File ( name = '/somePath/myScript.C' ) )
+</code>
+Furthermore it is very useful to run subjobs with different arguments for the script. For this you need the ArgSplitter:
+<code python>
+argList = [[1,2], [2,3]]
+argSplit = ArgSplitter( args = argList )
+j.splitter = argSplit
+</code>
+which will generate two subjobs with the arguments (1,2) and (2,3).
+A complete example therefore would look like:
+<code python>
+argList=[]
+for ia in range(1,5):
+    for ib in range(1,7):
+       argList.append([ ia, ib])
+j = Job()
+j.name = 'my great ROOT job'
+j.inputsandbox = [ File ( name = '/somePath/lib1.so'), File ( name = '/somePath2/lib2.so')]
+j.application = Root (
+    version = '5.34.00' ,
+    usepython = False ,
+    script = File ( name = '/somePath/myScript.C' ) )
+argSplit = ArgSplitter( args = argList )
+j.splitter = argSplit
+j.submit()
+</code>
+  *Note: Due to a weird bug (1.8.2012) 0 is not allowed as an argument...
+  *Note: When submitting a ROOT-job to the LSF, make sure the Ganga version in the shell you submit the ganga-job from and the one requested in ''j.application'' agree. You can set the ROOT version for Ganga in the shell with (for example): ''SetupProject Ganga v508r6 ROOT -v 5.34.00''
 ===== Forcing a job to run on a specific site =====
 For this, do:
@@ Line 399: / Line 463: @@
 </code>
 Note that this is not the desired behaviour, as it is not what grid computing is about, but can be useful sometimes.
+===== Job fails after many reschedulings =====
+If your grid-certificate is close to expiring, it is possible that no proxy can be created for the full job duration. If you already have a new grid cert, you have to upload it to Dirac. This is done via:
+<code python>
+SetupProject LHCbDirac
+dirac-proxy-init -g lhcb_user
+</code>
+===== Bookkeeping information within Ganga =====
+A simple method that can be added to the ~/.ganga.py to access information directly from the BK can be seen below:
+<code python>
+def getBKInfo ( evttype )  :
+    from subprocess import Popen, PIPE
+    serr = open ( '/dev/null' )
+    pipe = Popen ( [ 'get_bookkeeping_info'  , str(evttype) ] ,
+                   env    = os.environ ,
+                   stdout = PIPE       ,
+                   stderr = serr       )
+    stdout = pipe.stdout
+    ts = {}
+    result = {}
+    for line in stdout :
+        try :
+            value = eval ( line )
+        except :
+            continue
+        if not isinstance ( value    , tuple ) : continue
+        if not isinstance ( value[0] , str   ) : continue
+        if not isinstance ( value[1] , str   ) : continue
+        if not isinstance ( value[2] , str   ) : continue
+        if result.has_key ( value[0] ) : continue
+        result [ value[0] ] = value[1:]
+    return result
+</code>
+In this case two additional files 'get_bookkeeping_info' and 'dirac-bookkeeping-get-prodinfo-eventtype.py' are required to be saved locally in your ~/bin/ directory.
+More info can be found here: [[https://groups.cern.ch/group/lhcb-bender/Lists/Archive/DispForm.aspx?ID=551]]
+and here: [[https://groups.cern.ch/group/lhcb-bender/Lists/Archive/DispForm.aspx?ID=695]]
 ===== Help =====
 To see the documentation, type ''help()'' in the interactive mode. To get help about a specific topic, e.g. Dirac, type ''help(Dirac)'' in the interactive mode.
-===== The n Commandments when working with Ganga =====
+===== The N Commandments when working with Ganga =====
   *Thou shall be patient.
   *Thou shall never use PFNs when thou needest LFNs.
-  *Thou shall write an email to ''lhcb-distributed-analysis@cern.ch'', if thou art desparate. Thou shall wait patiently for a reply and thy wisdom will flourish.
+  *Thou shall write an email to ''lhcb-distributed-analysis@cern.ch'', if thou art desperate. Thou shall wait patiently for a reply and thy wisdom will flourish.
   *Thou shall start a test-job before running the full job. The ways of making mistakes are manifold.
   *Thou shall always check if thy output will be bearable for thy quota.
@@ Line 413: / Line 524: @@
 Information for using ganga in LHCb (and where I stole most of the information from): [[http://ganga.web.cern.ch/ganga/user/html/LHCb/]]
+Ganga/Dirac mailing list archive: [[https://groups.cern.ch/group/lhcb-distributed-analysis/Lists/Archive/100.aspx]]

Physik-Institut

LHCb Wiki Pages

Differences

Physik-Institut

LHCb Wiki Pages

User Tools

Site Tools

Differences

Page Tools