[WIP] Theory covariance matrix plots in validphys2

Created by: RosalynLP

Hello, as discussed in the phone conference just now Cameron and I have added some features to results.py and plots.py in validphys2 to reproduce the plots in "RP+CV notes" in the wiki.

There are a few changes we're planning to make in the near future:

• Implement calculation of experiment and theory correlation matrices by taking in the corresponding covariance matrices as an input. Currently these are redefined within the correlation matrix action which is inefficient.

• Remove unnecessary instances of t0 checks. These appear in the definitions of theory covariance matrices in an analogy with the pre-existing experiment covariance matrix action, but we think it shouldn't be necessary. However, we don't understand exactly what this bit of the code is doing.

• Introduce user-controlled (as in inputted from runcard) limits for the colour bars in the matrix heat plots, and a legend which displays these values. Currently these limits are set values in the code.

• Add experiment and data set labels to the matrix plots so it's clearer which elements correspond to which data sets

• In the current output there seems to be a bug relative to some point in the past, where the theory covariance matrix has some very high valued elements:

This means the version in this pull request has output with some discrepancy compared the plots in the wiki. We are working on fixing this currently.

requested review from @enocera

requested review from @enocera

requested review from @enocera

Created by: Zaharid

Thanks! This looks fine from a quick look. One high level suggestion would be to leave the stuff that is going to be generally useful (like the correlation matrix) in results.py, and move everything more specific to the theory covariance matrix to a dedicated module. Experience with the alpha_s determination suggests that it is going to get messy eventually, so better to keep it contained.

Something similar applies to the runcards. We could (and should) have one example, but I would suggest using the reports repository: http://pcteserver.mi.infn.it/~nnpdf/validphys-reports/ because again there will need to be a bazillion variations of them eventually. Also, this makes it easy to show around the results.

Created by: Zaharid

Review: Commented

Will do more comments eventually.

Created by: Zaharid

The color map you use for the matrix plots is nice as a "sequential" color map. However here we probably want to have a "divergent" one, that is cantered at zero. The one I like is Spectral_r. See

https://github.com/NNPDF/nnpdf/blob/800ac01bab549bbf808f1fdb05430c3d3cb241a4/validphys2/src/validphys/plots.py#L1120

Created by: Zaharid

Maybe you can trying to abstract all these away by defining a more generic matrix plotting function in plotutils.py and calling it with different arguments?

Created by: RosalynLP

Good idea we'll add that to the list of stuff to change

Created by: RosalynLP

Thanks, I've changed the colourmaps and will push the changes once I've seen the output of the runcard! At the moment it runs quite slow which may be in part due to the way we've continually calculated all the covariance/correlation matrices from scratch. It'll be interesting to see what effect there will be when we improve this.

Created by: RosalynLP

If people are interested, the plot above now looks like:

Created by: Zaharid

The covariance matrices already come with the names of the experiments and datasets (the experiments_index thing), so it shouldn't be too hard to put that in matplotlib. It will probably be hard to make it look nice, and will involve fiddling with the relative size of the various elements.

Created by: Zaharid

Also, ideally we would like to have even more levels, with the kinematics of each dataset. But let's leave that for later.

Created by: Zaharid

Could you upload a few example reports to the vp repository?