| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| limits:limits [2018/05/04 16:26] – [Quick setup] iwn | limits:limits [2018/10/04 16:41] (current) – [Limits with the Higgs Combined Tool] iwn |
|---|
| |
| The **Combined Tool** by the [[https://twiki.cern.ch/twiki/bin/view/CMS/HiggsWG/HiggsCombination|Higgs Combination Group]] provides the ''combine'' command which runs different statistical methods with RooStats for some input, e.g. a text file (datacard) containing information on the systematic uncertainties, data and processes, [[limits:limits#Simple example: Running the combine tool|like the example below]]. It is extensively discussed here: | The **Combined Tool** by the [[https://twiki.cern.ch/twiki/bin/view/CMS/HiggsWG/HiggsCombination|Higgs Combination Group]] provides the ''combine'' command which runs different statistical methods with RooStats for some input, e.g. a text file (datacard) containing information on the systematic uncertainties, data and processes, [[limits:limits#Simple example: Running the combine tool|like the example below]]. It is extensively discussed here: |
| * https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit | * TWiki documentation: https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit |
| * https://cms-hcomb.gitbooks.io/combine/content/ | * GitBooks documentation: https://cms-hcomb.gitbooks.io/combine/content/ |
| | * Hypernews forum: https://hypernews.cern.ch/HyperNews/CMS/get/higgs-combination |
| |
| [[https://indico.cern.ch/event/577649/#day-2016-11-30|These]] are useful **tutorials** on the Combine Tool from a 2016 CERN workshop (slides plus video). It may be interesting to do the exercises on [[https://indico.cern.ch/event/577649/sessions/212214/attachments/1378209/2093750/combine_SWAN_instructions.pdf|SWAN]]. Otherwise, take a look at [[https://github.com/nucleosynthesis/HiggsAnalysis-CombinedLimit/blob/combine_tutorial_SWAN/combine_tutorials_2016/combine_intro/python_CombineIntro.ipynb|these examples on GitHub]]. | [[https://indico.cern.ch/event/577649/#day-2016-11-30|These]] are useful **tutorials** on the Combine Tool from a 2016 CERN workshop (slides plus video). It may be interesting to do the exercises on [[https://indico.cern.ch/event/577649/sessions/212214/attachments/1378209/2093750/combine_SWAN_instructions.pdf|SWAN]]. Otherwise, take a look at [[https://github.com/nucleosynthesis/HiggsAnalysis-CombinedLimit/blob/combine_tutorial_SWAN/combine_tutorials_2016/combine_intro/python_CombineIntro.ipynb|these examples on GitHub]]. |
| | |
| | Many other **examples** can be found on [[https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookExercisesCMSDataAnalysisSchool|this TWiki page for the CMSDAS school]]. |
| |
| Description of the statistical methods at CMS: | Description of the statistical methods at CMS: |
| ===== Quick setup ===== | ===== Quick setup ===== |
| |
| Get **CMSSW** for the Combined Tool: | Latest instructions to get ''combine'' can be found [[https://cms-hcomb.gitbooks.io/combine/content/part1/#for-end-users-that-dont-need-to-commit-or-do-any-development|in the manual]] and [[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#ROOT606_SLC6_CMSSW_8_1_X|on the TWiki]]. In summary: Get **CMSSW** for the Combined Tool: |
| |
| <code bash> | <code bash> |
| export SCRAM_ARCH=slc6_amd64_gcc491 | export SCRAM_ARCH=slc6_amd64_gcc530 |
| cmsrel CMSSW_8_1_0 | cmsrel CMSSW_8_1_0 |
| cd CMSSW_8_1_0/src | cd CMSSW_8_1_0/src |
| </code> | </code> |
| |
| (''CMSSW_7_4_7'' is outdated.) Get **Combined Tool**: | (''combine'' in ''CMSSW_7_4_7'' is outdated.) Get **Combined Tool**: |
| |
| <code bash> | <code bash> |
| git clone https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit.git HiggsAnalysis/CombinedLimit | git clone https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit.git HiggsAnalysis/CombinedLimit |
| cd HiggsAnalysis/CombinedLimit | cd HiggsAnalysis/CombinedLimit |
| | git fetch origin |
| | git checkout v7.0.8 |
| | scramv1 b clean; scramv1 b |
| </code> | </code> |
| |
| |
| <code bash> | <code bash> |
| source /afs/cern.ch/cms/cmsset_default.sh | source $VO_CMS_SW_DIR/cmsset_default.sh |
| export SCRAM_ARCH=slc6_amd64_gcc491 | export SCRAM_ARCH=slc6_amd64_gcc530 |
| cd ~/phase2/CMSSW_7_4_7/src/ | cd ~/phase2/CMSSW_8_0_1/src/ |
| eval `scram runtime -sh` | cmsenv |
| </code> | </code> |
| |
| ===== Simple example: Running the combine tool ===== | ===== Simple example: Running the combine tool ===== |
| |
| |
| The combine tool has many more [[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#How_to_run_the_tool|methods]] and options (type ''%%combine --help%%''). | The combine tool has many more [[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#How_to_run_the_tool|methods]] and options (type ''%%combine --help%%''). |
| If you are interested in the **expected significance**, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Maximum_likelihood_fits_and_diag|MaxLikelihoodFit]]''. This can method also can be used to extract the **observed cross section**. | If you are interested in the **expected significance**, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Maximum_likelihood_fits_and_diag|MaxLikelihoodFit]]'' (''FitDiagnositcs'' in ''CMSSW_8_1_0''). This can method also can be used to extract the **observed cross section**. |
| If you are interested in only an //approximation// of the **expected number of signal events** (**yield**) with the maximum likelihood fit, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Computing_the_observed_limit_wit|ProfileLikelihood]]''. For the **upper limits** however, you always should use ''Asymptotic'' instead. | If you are interested in only an //approximation// of the **expected number of signal events** (**yield**) with the maximum likelihood fit, use ''[[https://twiki.cern.ch/twiki/bin/viewauth/CMS/SWGuideHiggsAnalysisCombinedLimit#Computing_the_observed_limit_wit|ProfileLikelihood]]''. For the **upper limits** however, you always should use ''Asymptotic'' instead. |
| |
| ===== Brazilian plot ===== | ===== Brazilian plot ===== |
| |
| To make a Brazilian plot, you can make a simple python script that automatically creates the datacards for different values of some parameter like the invariant mass. [[limits:brazilianplotexample|This example]] shows scan for the background uncertainty. | To make a Brazilian plot, you can make a simple python script that automatically creates the datacards for different values of some parameter like the invariant mass. [[limits:brazilianplotexample|This example]] shows scan for the background uncertainty. Guidelines for a CMS-style limit plot can be found [[https://ghm.web.cern.ch/ghm/plots/|here]] and [[https://twiki.cern.ch/twiki/bin/view/CMS/Internal/FigGuidelines#Convention_for_Brazilian_flag_pl|here]]. |
| {{ limits:upperlimit_brazilianplot.png ? 400 }} | {{ limits:upperlimit_brazilianplot.png ? 400 }} |
| |
| |
| ===== Other ===== | ===== Other ===== |
