Covmat speedup
Created by: wilsonmr
Sorry I got over excited and wanted to try and get this done, because then you can get going on #866 asap.
As discussed in a few places, reformatting slightly the coredata.CommonData class looked like we could get some speedup when constructing covmat:
Before:
In [1]: from validphys.loader import Loader
In [2]: from validphys.commondataparser import load_commondata
In [3]: from validphys.covmats import covmat_from_systematics
In [4]: l = Loader()
In [5]: cd = l.check_commondata("NMC")
In [6]: lcd = load_commondata(cd)
In [7]: %timeit covmat_from_systematics(lcd)
249 ms ± 1.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)After:
In [1]: from validphys.loader import Loader
In [2]: from validphys.commondataparser import load_commondata
In [3]: from validphys.covmats import covmat_from_systematics
In [4]: l = Loader()
In [5]: cd = l.check_commondata("NMC")
In [6]: lcd = load_commondata(cd)
In [7]: %timeit covmat_from_systematics(lcd)
5.12 ms ± 11.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)@siranipour there are a few things I didn't check very thoroughly. For example whether the docstring makes sense now. I also think that there was no other dependence of the sys_errors from before, but you're the expert so tell me if I broke something.
What do you think of this layout? I think it's fairly readable, but maybe I made it really ugly.
I think the loops we do are ok because although it's a nested for loop it only goes over maximum of like 12 iterations per dataset (every single type of uncertainty for both mult and add)
The worst part is handling the case where there are no special correlated systematics for a dataset, but this was the best I could think of, maybe there is a better way to handle this.
I think that the CommonData properties I added could use the common sys_table, perhaps this can be set at point of loading?