Fixed inconsistencies between systype files generated by buildmaster …

This PR addresses #1265 (closed) . I have found the following inconsistencies between the output of buildmaster and the files stored in nnpdfcpp.

A) DATA files: no inconsistencies, except for ZpT data (see below) and for the fact that a slightly different formatting is used sometimes (possibly due to the generation of the DATA files on different machines). -> No action taken.

B) SYSTYPE files: I have found inconsistencies for the following experiments:

• NMC: the four last uncertainties should be MULT, as in the SYSTYPE file stored in nnpdfcpp -> corrected in buildmaster;
• ATLAS_DY_2D_8TEV_LOWMASS: there is no luminosity uncertainty (the cross section is normalised); the SYSTYPE file in nnpdfcpp is correct -> corrected buildmaster;
• ATLAS_WP_JET_8TEV_PT and ATLAS_WM_JET_8TEV_PT: the non-perturbative uncertainty should be SKIP, as non-perturbative corrections are not applied; the SYSTYPE file in nnpdfcpp is correct -> corrected buildmaster;
• ATLAS_TTB_DIFF_8TEV_LJ_TPTNORM, ATLAS_TTB_DIFF_8TEV_LJ_TRAPNORM, ATLAS_TTB_DIFF_8TEV_LJ_TTRAPNORM and ATLAS_TTB_DIFF_8TEV_LJ_TTMNORM: an uncertainty was incorrectly assumed as MULT while it should be ADD; the implementation in buildmaster is correct -> corrected files stored in nnpdfcpp.
• ATLASZPT7TEV, ATLASZPT8TEVMDIST, ATLASZPT8TEVYDIST, CMSZDIFF12: an extra theoretical uncertainty is appended in the DATA and SYSTYPE files stored in nnpdfcpp. These files are generated from the buildmaster output by running the script process_zpt.sh in nnpdf/nnpdfcpp/data/commondata/31systematics. Therefore the buildmaster and nnpdfcpp files ought to be different. This way of processing the files for these datasets is a remnant of NNPDF3.1. -> No action taken (except to check that the script reproduces the files stored in nnpdfcpp from the output of buildmaster).

Note that I have checked only the datasets that we used or that are going to be used for NNPDF4.0. There might be inconsistencies in deprecated/outdated files. In this respect, I think that the best thing to do is to remove them, a separate issue on a general cleanup is perhaps advisable before making the code public.

Note also that I've realised that the ATLASPHT15 and ATLASPHT12 datasets specified in the template runcards and used so far should be replaced by the variants ATLASPHT15_SF and ATLASPHT12_SF, which are the only ones produced by buildmaster.

In summary, all the inconsistencies had to be fixed at the level of buildmaster, except for the ATLAS photon data and for the ATLAS 8 TeV top-pair differential data. These changes seem however inconsequential if one compares the chi2 obtained after these changes (see https://vp.nnpdf.science/diPHOGXBToSdC8K02654sg==) with the chi2 obtained before these changes (see https://vp.nnpdf.science/NQzde1SETUuB-Ma677wMrg==). I therefore conclude that the weighted fits do not need to be repeated.

Finally, note that his PR also includes an update of the template runcards, in which the correct denomination of the surviving ATLASPHT15_SF dataset has been updated and the datasets that we decided to discard from the baseline fit (as an outcome of the recent weighted fit campaign) have been removed.

requested review from @enocera

requested review from @enocera

@Zaharid Once this is reviewed and merged an updated tag/environment should be used to run the (final?) batch of NNPDF4.0 fits.

Created by: Zaharid

This PR addresses #1265 (closed) . I have found the following inconsistencies between the output of buildmaster and the files stored in nnpdfcpp.

B) SYSTYPE files: I have found inconsistencies for the following experiments:

* `NMC`: the four last uncertainties should be `MULT`, as in the `SYSTYPE` file stored in `nnpdfcpp` -> corrected in `buildmaster`;

* `ATLAS_DY_2D_8TEV_LOWMASS`: there is no luminosity uncertainty (the cross section is normalised); the `SYSTYPE` file in `nnpdfcpp` is correct -> corrected `buildmaster`;

* `ATLAS_WP_JET_8TEV_PT` and `ATLAS_WM_JET_8TEV_PT`: the non-perturbative uncertainty should be `SKIP`, as non-perturbative corrections are not applied; the `SYSTYPE` file in `nnpdfcpp` is correct -> corrected `buildmaster`;

How come the files were inconsistent? Is it that the right version of buildmaster was never committed?

This PR addresses #1265 (closed) . I have found the following inconsistencies between the output of buildmaster and the files stored in nnpdfcpp. B) SYSTYPE files: I have found inconsistencies for the following experiments:

* `NMC`: the four last uncertainties should be `MULT`, as in the `SYSTYPE` file stored in `nnpdfcpp` -> corrected in `buildmaster`;

* `ATLAS_DY_2D_8TEV_LOWMASS`: there is no luminosity uncertainty (the cross section is normalised); the `SYSTYPE` file in `nnpdfcpp` is correct -> corrected `buildmaster`;

* `ATLAS_WP_JET_8TEV_PT` and `ATLAS_WM_JET_8TEV_PT`: the non-perturbative uncertainty should be `SKIP`, as non-perturbative corrections are not applied; the `SYSTYPE` file in `nnpdfcpp` is correct -> corrected `buildmaster`;

How come the files were inconsistent? Is it that the right version of buildmaster was never committed?

I'm sorry to say the answer to the second question is yes. Concerning the new experiments ATLAS_DY_2D_8TEV_LOWMASS, ATLAS_WP_JET_8TEV_PT and ATLAS_WM_JET_8TEV_PT it's entirely my fault; concerning NMC, I suspect that the change was done on the SYSTYPE file stored in nnpdfcpp without updating the buildmaster implementation. Anyway that should have happened years ago, because we haven't touched NMC for a long time.

added urgent label

Created by: voisey

Hi @enocera, in your summary you say that there is no luminosity uncertainty for this dataset since it is normalised, but then shouldn't this uncertainty be removed entirely rather than being made UNCORR?

I was not clear: there is a 1.9% uncorrelated uncertainty, but this is not a luminosity uncertainty, hence the uncertainty must remain there, but with the correct label UNCORR.

Created by: voisey

Okay, that makes sense then, thank you

Created by: voisey

Review: Approved

Created by: voisey

I've been through all of the changes in this PR and everything seems consistent with @enocera's summary, so from my perspective this looks good to go.

Created by: Zaharid

@enocera Is this change covered by something?

git --no-pager diff ../nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
diff --git a/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat b/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
index fe484c6cb..49b5f4e61 100644
--- a/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
+++ b/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
@@ -1,4 +1,4 @@
-406
+404
 1    MULT    CORR
 2    MULT    CORR
 3    MULT    CORR
@@ -403,5 +403,3 @@
 402    MULT    CORR
 403    MULT    ATLASLUMI11
 404    MULT    ATLAS1JET11_NP_err
-405    MULT    SKIP
-406    MULT    SKIP

Created by: Zaharid

ATLAS_WP_JET_8TEV_PT and ATLAS_WM_JET_8TEV_PT: the non-perturbative uncertainty should be SKIP, as non-perturbative corrections are not applied; the SYSTYPE file in nnpdfcpp is correct -> corrected buildmaster;

This is what I get if I copy the buildmaster file to nnpdfcpp

--- a/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS_WM_JET_8TEV_PT_DEFAULT.dat
+++ b/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS_WM_JET_8TEV_PT_DEFAULT.dat
@@ -122,4 +122,4 @@
 121    MULT    UNCORR
 122    MULT    UNCORR
 123    MULT    UNCORR
-124    MULT    SKIP
+124    MULT    CORR

@enocera Is this change covered by something?

git --no-pager diff ../nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
diff --git a/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat b/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
index fe484c6cb..49b5f4e61 100644
--- a/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
+++ b/nnpdfcpp/data/commondata/systypes/SYSTYPE_ATLAS1JET11_SF_DEFAULT.dat
@@ -1,4 +1,4 @@
-406
+404
 1    MULT    CORR
 2    MULT    CORR
 3    MULT    CORR
@@ -403,5 +403,3 @@
 402    MULT    CORR
 403    MULT    ATLASLUMI11
 404    MULT    ATLAS1JET11_NP_err
-405    MULT    SKIP
-406    MULT    SKIP

No, it isn't. However this falls under this category:

Note that I have checked only the datasets that we used or that are going to be used for NNPDF4.0. There might be inconsistencies in deprecated/outdated files. In this respect, I think that the best thing to do is to remove them, a separate issue on a general cleanup is perhaps advisable before making the code public.

This dataset is by now obsolete, as it is now replaced by ATLAS1JET11_NEWSCALE (in fits where single-inclusive jets are used instead of dijets). The NNPDF3.1 results are reproduced by using ATLAS1JET11.

@Zaharid Are you sure about the discrepancy in the ATLAS_WM_JET_8TEV_PT? I don't see it. And here is where I thought I fixed it.

Created by: Zaharid

No, not sure. I think I forgot to type make install.

Created by: Zaharid

There is a set called LHCB_WENU_8TEV_R that has a factor 1000 difference in kin3 (I don't see it included by default but sounds similar to LHCB_WENU_8TEV_A).

Created by: Zaharid

I guess this doesn't really close #1265 (closed) and instead we need another PR for that.

There is a set called LHCB_WENU_8TEV_R that has a factor 1000 difference in kin3 (I don't see it included by default but sounds similar to LHCB_WENU_8TEV_A).

True. The inconsistency comes from the fact that kin3 should be in GeV, while the file stored in nnpdfcpp had it in TeV. This is fixed.

I guess this doesn't really close #1265 (closed) and instead we need another PR for that.

I agree. I've opened #1275 (closed) .

Created by: Zaharid

At this point merging this is probably better than not.

Created by: Zaharid

Review: Approved

merged manually