Use python covmat in multiclosure tests
Created by: scarlehoff
This closes issue #1486 (closed) while dealing with another of the not-trivial cases of #1475 (closed). And the last preventing me from working on #1475 (closed) because I've decided not to touch the filter at this stage.
This "fix" is very tricky because there are no examples or tests where I can check what the intended behavior is exactly. Thanks to @tgiani I have at least a runcard that uses the function:
fit: 210326-mw-001
theoryid: 200
use_cuts: internal
q2min: 3.49 # Q2 minimum
w2min: 12.5 # W2 minimum
dataset_inputs:
# ATLAS
- {dataset: DYE906R_dw_ite, frac: 0.75, cfac: [ACC,QCD]}
- { dataset: ATLASWZRAP11CF, frac: 0.75, cfac: [QCD] } # N
meta:
author: TG
keywords: [test]
title: Test plot_data_fits_bias_variance
diag_ns:
- diagonal_basis: True
- diagonal_basis: False
template_text: |
# Results with 25 fits
## Central diff histogram
{@with diag_ns@}
# Diagonal: {@diagonal_basis@}
{@plot_data_fits_bias_variance@}
{@endwith@}
use_t0: true
fits:
- 210326-mw-001
- 210326-mw-002
actions_:
- report(main=True)
Honestly, I would've like not to have modified config.py
or force people to have to use use_t0
when before it wasn't the case, however due to https://github.com/NNPDF/nnpdf/blob/b44ae914ef494049f9456c8c0fdd267e8386446e/validphys2/src/validphys/closuretest/multiclosure.py#L49 just calling dataset_inputs_t0_covmat_from_systematics
is non-trivial, doable, but complicates quite a bit the logic (unless there's some shortcut I haven't found).
Fix
Just using dataset_inputs_t0_covmat_from_systematics
. I've checked that for the above runcard the results are completely equivalent. This also simplifies the function since it no longer have to take care of doing the loading of the data and setting the t0 set etc (which included a copy not sure why)
Backwards compatibility
Since this breaks previously working runcards (in that now the user has to add one extra flag) and in that the usage of the t0 covmat is not happening in the dark in some function but done by validphys I've added two checks:
-
use_t0
just telling the user the runcard should be updated - Ensure that the user doesn't mistakenly set a
t0set
different from themulticlosure_law
.
Tests and examples
It would be nice if someone could add tests to this module which is currently lacking. I honestly don't know what to test other than it runs. I'll add the runcard @tgiani gave me but for the tests someone else should do another PR that takes care of this because I'm mostly guessing how things should work so I don't think I'm qualified to write a test specifying how really things should work.
Anyway this is ready for review.