| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| limits:limits [2017/08/09 01:35] – [Limits with the Higgs Combined Tool] iwn | limits:limits [2018/10/04 16:41] (current) – [Limits with the Higgs Combined Tool] iwn |
|---|
| ====== Limits with the Higgs Combined Tool ====== | ====== Limits with the Higgs Combined Tool ====== |
| |
| The **Combined Tool** by the [[https://twiki.cern.ch/twiki/bin/view/CMS/HiggsWG/HiggsCombination|Higgs Combination Group]] provides the ''combine'' command which runs different statistical methods with RooStats for some input, e.g. a text file (datacard) containing information on the systematic uncertainties, data and processes, [[limits:limits#Simple example: Running the combine tool|like the example below]]. It is extensively discussed here: https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit | The **Combined Tool** by the [[https://twiki.cern.ch/twiki/bin/view/CMS/HiggsWG/HiggsCombination|Higgs Combination Group]] provides the ''combine'' command which runs different statistical methods with RooStats for some input, e.g. a text file (datacard) containing information on the systematic uncertainties, data and processes, [[limits:limits#Simple example: Running the combine tool|like the example below]]. It is extensively discussed here: |
| | * TWiki documentation: https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit |
| | * GitBooks documentation: https://cms-hcomb.gitbooks.io/combine/content/ |
| | * Hypernews forum: https://hypernews.cern.ch/HyperNews/CMS/get/higgs-combination |
| | |
| | [[https://indico.cern.ch/event/577649/#day-2016-11-30|These]] are useful **tutorials** on the Combine Tool from a 2016 CERN workshop (slides plus video). It may be interesting to do the exercises on [[https://indico.cern.ch/event/577649/sessions/212214/attachments/1378209/2093750/combine_SWAN_instructions.pdf|SWAN]]. Otherwise, take a look at [[https://github.com/nucleosynthesis/HiggsAnalysis-CombinedLimit/blob/combine_tutorial_SWAN/combine_tutorials_2016/combine_intro/python_CombineIntro.ipynb|these examples on GitHub]]. |
| | |
| | Many other **examples** can be found on [[https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookExercisesCMSDataAnalysisSchool|this TWiki page for the CMSDAS school]]. |
| |
| Description of the statistical methods at CMS: | Description of the statistical methods at CMS: |
| * Cowan et. al (2013), "Asymptotic formulae for likelihood-based tests of new physics",[[https://arxiv.org/abs/1007.1727|arXiv:1007.1727]]. | * Cowan et. al (2013), "Asymptotic formulae for likelihood-based tests of new physics",[[https://arxiv.org/abs/1007.1727|arXiv:1007.1727]]. |
| * [[https://indico.cern.ch/event/173726/|This four-part lecture]] by Glen Cowan covers a more **theoretical overview** of statistical methods used in Particle Physics. | * [[https://indico.cern.ch/event/173726/|This four-part lecture]] by Glen Cowan covers a more **theoretical overview** of statistical methods used in Particle Physics. |
| * [[https://indico.cern.ch/event/577649/#day-2016-11-30|Useful tutorials]] on the Combine Tool from a 2016 CERN workshop (slides plus video). It may be interesting to log into [[https://indico.cern.ch/event/577649/sessions/212214/attachments/1378209/2093750/combine_SWAN_instructions.pdf|SWAN]]. Otherwise, take a look at [[https://github.com/nucleosynthesis/HiggsAnalysis-CombinedLimit/blob/combine_tutorial_SWAN/combine_tutorials_2016/combine_intro/python_CombineIntro.ipynb|these examples on GitHub]]. | |
| |
| **CombineHarvester** is a tool to manage datacards: [[https://github.com/cms-analysis/CombineHarvester|its GitHub repository]] and [[http://cms-analysis.github.io/CombineHarvester/index.html|the full documentation]]. | **CombineHarvester** is a tool to manage datacards: [[https://github.com/cms-analysis/CombineHarvester|its GitHub repository]] and [[http://cms-analysis.github.io/CombineHarvester/index.html|the full documentation]]. |
| ===== Quick setup ===== | ===== Quick setup ===== |
| |
| Get **CMSSW** for the Combined Tool: | Latest instructions to get ''combine'' can be found [[https://cms-hcomb.gitbooks.io/combine/content/part1/#for-end-users-that-dont-need-to-commit-or-do-any-development|in the manual]] and [[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#ROOT606_SLC6_CMSSW_8_1_X|on the TWiki]]. In summary: Get **CMSSW** for the Combined Tool: |
| |
| <code bash> | <code bash> |
| export SCRAM_ARCH=slc6_amd64_gcc491 | export SCRAM_ARCH=slc6_amd64_gcc530 |
| cmsrel CMSSW_7_4_7 | cmsrel CMSSW_8_1_0 |
| cd CMSSW_7_4_7/src | cd CMSSW_8_1_0/src |
| cmsenv | cmsenv |
| </code> | </code> |
| |
| Get **Combined Tool**: | (''combine'' in ''CMSSW_7_4_7'' is outdated.) Get **Combined Tool**: |
| |
| <code bash> | <code bash> |
| git clone https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit.git HiggsAnalysis/CombinedLimit | git clone https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit.git HiggsAnalysis/CombinedLimit |
| cd HiggsAnalysis/CombinedLimit | cd HiggsAnalysis/CombinedLimit |
| | git fetch origin |
| | git checkout v7.0.8 |
| | scramv1 b clean; scramv1 b |
| </code> | </code> |
| |
| With every new session, set up the environment with | With every new session in PSI Tier3, set up the environment with |
| |
| <code bash> | <code bash> |
| source /afs/cern.ch/cms/cmsset_default.sh | source $VO_CMS_SW_DIR/cmsset_default.sh |
| export SCRAM_ARCH=slc6_amd64_gcc491 | export SCRAM_ARCH=slc6_amd64_gcc530 |
| cd ~/phase2/CMSSW_7_4_7/src/ | cd ~/phase2/CMSSW_8_0_1/src/ |
| eval `scram runtime -sh` | cmsenv |
| </code> | </code> |
| |
| |
| ===== Simple example: Running the combine tool ===== | ===== Simple example: Running the combine tool ===== |
| |
| |
| The combine tool has many more [[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#How_to_run_the_tool|methods]] and options (type ''%%combine --help%%''). | The combine tool has many more [[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#How_to_run_the_tool|methods]] and options (type ''%%combine --help%%''). |
| If you are interested in the **expected significance**, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Maximum_likelihood_fits_and_diag|MaxLikelihoodFit]]''. This can method also can be used to extract the **observed cross section**. | If you are interested in the **expected significance**, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Maximum_likelihood_fits_and_diag|MaxLikelihoodFit]]'' (''FitDiagnositcs'' in ''CMSSW_8_1_0''). This can method also can be used to extract the **observed cross section**. |
| If you are interested in only an //approximation// of the **expected number of signal events** (**yield**) with the maximum likelihood fit, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Computing_the_observed_limit_wit|ProfileLikelihood]]''. For the **upper limits** however, you always should use ''Asymptotic'' instead. | If you are interested in only an //approximation// of the **expected number of signal events** (**yield**) with the maximum likelihood fit, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Computing_the_observed_limit_wit|ProfileLikelihood]]''. For the **upper limits** however, you always should use ''Asymptotic'' instead. |
| |
| ===== Brazilian plot ===== | ===== Brazilian plot ===== |
| |
| To make a Brazilian plot, you can make a simple python script that automatically creates the datacards for different values of some parameter like the invariant mass. [[limits:brazilianplotexample|This example]] shows scan for the background uncertainty. | To make a Brazilian plot, you can make a simple python script that automatically creates the datacards for different values of some parameter like the invariant mass. [[limits:brazilianplotexample|This example]] shows scan for the background uncertainty. Guidelines for a CMS-style limit plot can be found [[https://ghm.web.cern.ch/ghm/plots/|here]] and [[https://twiki.cern.ch/twiki/bin/view/CMS/Internal/FigGuidelines#Convention_for_Brazilian_flag_pl|here]]. |
| {{ limits:upperlimit_brazilianplot.png ? 400 }} | {{ limits:upperlimit_brazilianplot.png ? 400 }} |
| |
| |
| ===== Other ===== | ===== Other ===== |
