Skip to content

Fix overfitmetric in comparefits

Emanuele Roberto Nocera requested to merge fix_for_overfitmetric into master

Created by: scarlehoff

I think due to this collect here https://github.com/NNPDF/nnpdf/blob/ec6cbaf972240de035b60e5321a4c406e36f78da/validphys2/src/validphys/overfit_metric.py#L23

The overfit metric in the comparefits report gets all predictions from both pdf, i..e, in this function https://github.com/NNPDF/nnpdf/blob/ec6cbaf972240de035b60e5321a4c406e36f78da/validphys2/src/validphys/overfit_metric.py#L51

preds is not just a prediction per dataset per replica but the union of what should be preds for both PDFs.

In other words, preds is a list of size datasets*2 where the first half are the predictions for one PDF and the second one for the other. The current comes first so if the current fit has more replicas than the reference it will work, but the result for he reference will probably be wrong.

This is very obvious if one tries to do a comparefits in which both PDF have a different number of replicas (and crucially, the current one has less than the reference) because it will fail.

Note: a better fix would be to modify the offending functions, but I wanted to have something working, so my fix has been overwrite the pdfs of the comparefits report.

Merge request reports

Loading