Fix overfitmetric in comparefits
Created by: scarlehoff
I think due to this collect here https://github.com/NNPDF/nnpdf/blob/ec6cbaf972240de035b60e5321a4c406e36f78da/validphys2/src/validphys/overfit_metric.py#L23
The overfit metric in the comparefits report gets all predictions from both pdf, i..e, in this function https://github.com/NNPDF/nnpdf/blob/ec6cbaf972240de035b60e5321a4c406e36f78da/validphys2/src/validphys/overfit_metric.py#L51
preds is not just a prediction per dataset per replica but the union of what should be preds for both PDFs.
In other words, preds
is a list of size datasets*2 where the first half are the predictions for one PDF and the second one for the other. The current
comes first so if the current
fit has more replicas than the reference
it will work, but the result for he reference
will probably be wrong.
This is very obvious if one tries to do a comparefits in which both PDF have a different number of replicas (and crucially, the current
one has less than the reference
) because it will fail.
Note: a better fix would be to modify the offending functions, but I wanted to have something working, so my fix has been overwrite the pdfs
of the comparefits
report.