[WIP] n3fit closure test
Created by: wilsonmr
Opening a PR so that we can review code which I am currently using to produce results
The PR opens with two main commits:
- add diagonal basis and closure
- add l2 regularizer
The diagonal basis is slightly redundant for the time being however since I added it and we might want to explore it in the future (and it features in similar areas to closure test changes) I left it in
The closure test is currently done in a way which I think should change although there are some positives to how it's done here which I think could be adapted to be satisfactory
At present:
- one runs vp-setupfit on the old style closure runcard, pseudodata is created from the selected PDF and stored in the directory
fitname/filter
. vp-setupfit then symlinksfitname
into<nnprofile specified results folder>/<fitname>
so it appears you have that fit locally. - n3fit sees a closure runcard and then adds to the config
use_fitcommondata: True;
fit: ` - commondata is loaded from the symlinked folder
- if the level 2 noise is on then it is added
My proposal to change this:
- There should exist set closure data associated with pseudofits or a very similar new mechanism for both level 0, and level 1 closure tests. The level 1 data could be several random seeds for several different generating PDFs, the level 0 would just be several level 0 PDFs
- vp-setupfit no longer needs to be ran (as with normal
n3fit
fits) and instead there would be an appropriate point to specifyuse_fitcommondata: True; fit: <pseudofitname>
in the runcard which would be the new standard way to do closure tests inn3fit
. In this way we can remove the horrible symlinking of a fit you are about to run into the results path and not reference itself in it's own config. - rest as before
The l2 regularizer is also not ideal at the moment - I added it with minimal brain power so copied the base_layer_selector
. the regularizer itself is backend dependent but perhaps it should have it's own module and the regularizer_selector is probably unneccessary with some thought.
I can split this up if it's deemed necessary I just wanted to have all of the stuff I changed with n3fit
in a PR. Finally I need to actually check all of this still works since I just spent ages moving it from the older n3fit
branch, but probably it is fine..