Use API in vp-nextfitruncard

Created by: wilsonmr

Just a small change on top of the really nice script from #863

I use the API so there is no duplicated code, this should make any future updating a bit easier.

This required making a small change to the provider to use an internal variable to update description

This also has the nice effect that if you don't own the fit, it gets downloaded automatically using the remote loader. (from my perspective - actually the main reason I changed this)

I also updated the seed generation to be able to generate any unsigned long int, since this is what is used by both our internal RNG and numpy - in practice this probably makes no difference but seemed a little neater

Let me know what you think!

BTW I checked by putting a random seed in that the output is identical doing it this way (since anyway the code is basically the same) but please feel free to check!

inadvertently closes #928 (closed)

requested review from @enocera

Created by: wilsonmr

@siranipour do you understand the error?

Created by: wilsonmr

@voisey I accidentally also fixed the issue you opened with the seeds, by the way that would also probably break nnfit because they both use long ints to set seeds

Created by: siranipour

@siranipour do you understand the error?

What error message are you referring to? I just checked and your modifications seem to be working fine

Created by: wilsonmr

the travis error on mac. When I run tests locally it works fine, but for some reason it breaks in conda-build

I don't really understand how the pseudodata functions work especially with your fancy parallelisation so I was wondering if the traceback meant anything to you?

Created by: siranipour

Oh yeah let me take a look for you. In the meantime I'm getting a loader test failure locally:

../validphys2/src/validphys/tests/test_loader.py:54: in test_rebuild_commondata_without_cuts
    t1 = kitable(cd, get_info(cd))
../validphys2/src/validphys/plotoptions/core.py:61: in get_info
    info = PlotInfo.from_commondata(data, cuts=cuts, normalize=normalize)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'validphys.plotoptions.core.PlotInfo'>
commondata = CommonDataSpec(datafile=PosixPath('/home/shayan/miniconda3/envs/nnpdf-dev/share/NNPDF/data/commondata/DATA_CMSTOPDIFF8...plotfiles=(PosixPath('/home/shayan/miniconda3/envs/nnpdf-dev/share/NNPDF/data/commondata/PLOTTINGTYPE_HQP_PTQ.yaml'),))
cuts = None, normalize = False

    @classmethod
    def from_commondata(cls, commondata, cuts=None, normalize=False):
    
        #The only reason to call the parser once per config file is to
        #give better error messages and stricter checks
        plot_params = ChainMap()
        if commondata.plotfiles:
            for file in commondata.plotfiles:
                with open(file) as f:
                    config = PlotConfigParser.from_yaml(f, commondata, cuts=cuts)
                try:
                    config_params = config.process_all_params()
                except ConfigError:
                    log.error(f"Error in plotting file: {file}")
                    raise
    
                plot_params = plot_params.new_child(config_params)
            if normalize and 'normalize' in plot_params:
                #We might need to use reportengine.namespaces.resolve here
                plot_params = plot_params.new_child(config_params['normalize'])
            if not 'dataset_label' in plot_params:
                log.warning(f"'dataset_label' key not found in {file}")
                plot_params['dataset_label'] = commondata.name
    
        else:
            plot_params = {'dataset_label':commondata.name}
    
        kinlabels = commondata.plot_kinlabels
        if 'kinematics_override' in plot_params:
            kinlabels = plot_params['kinematics_override'].new_labels(*kinlabels)
    
>       return cls(kinlabels=kinlabels, **plot_params)
E       TypeError: __init__() missing 2 required keyword-only arguments: 'experiment' and 'nnpdf31_process'

Are you not getting this?

Created by: wilsonmr

I'm not, is your commondata up to date? That looks like the kind of error when the metadata is out of date, so the PLOTTING* files don't have necessary keys in.

EDIT: and is fixed by recompiling code and reinstalling (sometimes I delete commondata directory in the install location to check that it definitely gets replaced)

Created by: siranipour

That's what I was thinking but reinstalling didn't fix it. However, deleting and then reinstalling commondata did! Many thanks.

I'll see what I can do re: this pseudodata error it's an odd one.

Created by: wilsonmr

weird that it is only on mac and weird that it appears to be intermittent since my other PR today passed: #930

when I say weird, I guess what I mean is completely consistent with the behaviour we have seen with the mac builds..

Created by: wilsonmr

Ok I think I can reproduce this only when I run conda-build locally - which is annoying because it takes a while but at least means I can see if I can work out what's happening

I'm going to open an issue for that because I think it's completely detached from this PR

Created by: voisey

Review: Commented

Very nice improvement, thanks @wilsonmr! And thanks for solving the issue as well

Created by: voisey

Review: Approved

Created by: Zaharid

LGTM!

Merged by: scarrazza at 2020-09-23 14:24:06 UTC

merged manually

closed

mentioned in issue #938 (closed)