Skip to content

Automatic port old commondata to the new format

Emanuele Roberto Nocera requested to merge automatic_port_old_commondata into new_commondata_collected

Created by: scarlehoff

This is the port of all the old commondata into the new format.

It is mostly working and it is mostly automated. However, before pushing the new set of data I need some input.

In the file buildmaster/old_new_porting_map.yml you can find the mapping that I'm using. The mapping (maps) datasets to:

  1. A new name
  2. The arxiv and journal reference (thanks @enocera !). There's also hepdata for some since I already had some from pinecards. But it might be misleading. Feel free to point out when this is the case and I will remove it.

ℹ HELP and FEEDBACK WANTED:

Please, review the file buildmaster/old_new_porting_map.yml it should be one of the ones at the top if you go to Files Changed.

If people could already go through the naming that would very helpful. Specially the choice of name for the observable.

And, please, check the energies for CHORUS / SLAC / NUTEV / etc. I've put the average energy as @enocera suggested, but not sure what to put for SLAC there.

For the positivity datasets, all of them are now part of the POS process and the NNPDF experiment. As far as I am aware, experiment and process for positivity datasets should not mean anything, but if it does somehow please let me know.

Anything else you find funny or think that should be changed, please point it out!

Thank you very much!

(note that the mapping should not include datasets already implemented)

Todo:

  • In the next few days I will push also the actual data, together with reports separated by experiment to check that everything works with the reader. For now I've only checked I could read / compute covmat / compute predictions. I want to check that the plotting also works.
  • I will add a legacy version to some of the dataset already implemented if necessary (or change the variant name to legacy, instead of bugged)
  • Still not sure how to deal with #1908 (closed), but I think I will add a fake dataset.
  • Go through the entire dataset, not only 4.0. The amount of metadata there will be of, err, lesser quality. But I think we can live with that.

Once everything is working I will remove all old commondata and the possibility of reading it up with validphys.

Merge request reports