Skip to content

Use python covmat in multiclosure tests

Emanuele Roberto Nocera requested to merge multiclosure_without_lhapdfset into master

Created by: scarlehoff

This closes issue #1486 (closed) while dealing with another of the not-trivial cases of #1475 (closed). And the last preventing me from working on #1475 (closed) because I've decided not to touch the filter at this stage.

This "fix" is very tricky because there are no examples or tests where I can check what the intended behavior is exactly. Thanks to @tgiani I have at least a runcard that uses the function:

fit: 210326-mw-001

theoryid: 200
use_cuts: internal
q2min: 3.49                        # Q2 minimum
w2min: 12.5                        # W2 minimum

dataset_inputs:
# ATLAS
- {dataset: DYE906R_dw_ite, frac: 0.75, cfac: [ACC,QCD]}
- { dataset: ATLASWZRAP11CF, frac: 0.75, cfac: [QCD] } # N

meta:
 author: TG
 keywords: [test]
 title: Test plot_data_fits_bias_variance

diag_ns:
 - diagonal_basis: True
 - diagonal_basis: False

template_text: |
 # Results with 25 fits

 ## Central diff histogram
 {@with diag_ns@}
 # Diagonal: {@diagonal_basis@}
 {@plot_data_fits_bias_variance@}
 {@endwith@}

use_t0: true

fits:
- 210326-mw-001
- 210326-mw-002

actions_:
 - report(main=True)

Honestly, I would've like not to have modified config.py or force people to have to use use_t0 when before it wasn't the case, however due to https://github.com/NNPDF/nnpdf/blob/b44ae914ef494049f9456c8c0fdd267e8386446e/validphys2/src/validphys/closuretest/multiclosure.py#L49 just calling dataset_inputs_t0_covmat_from_systematics is non-trivial, doable, but complicates quite a bit the logic (unless there's some shortcut I haven't found).

Fix

Just using dataset_inputs_t0_covmat_from_systematics. I've checked that for the above runcard the results are completely equivalent. This also simplifies the function since it no longer have to take care of doing the loading of the data and setting the t0 set etc (which included a copy not sure why)

Backwards compatibility

Since this breaks previously working runcards (in that now the user has to add one extra flag) and in that the usage of the t0 covmat is not happening in the dark in some function but done by validphys I've added two checks:

  • use_t0 just telling the user the runcard should be updated
  • Ensure that the user doesn't mistakenly set a t0set different from the multiclosure_law.

Tests and examples

It would be nice if someone could add tests to this module which is currently lacking. I honestly don't know what to test other than it runs. I'll add the runcard @tgiani gave me but for the tests someone else should do another PR that takes care of this because I'm mostly guessing how things should work so I don't think I'm qualified to write a test specifying how really things should work.

Anyway this is ready for review.

Merge request reports

Loading