Skip to content

Merge prefactors into single layer

Emanuele Roberto Nocera requested to merge parallel-prefactor into master

Created by: APJansen

Join the preprocessing layers into one.

This also required changing the way weights are extracted and set, as a replica now is only a single slice of a weight tensor. Locally for me the way I implemented it is giving errors that seem specific to Mac, running on the GPU.

Status: Done, ready for review

  • I started changing stuff in vpinterface, but I think what I should do instead is similar to what I did before with the PDF as a whole: use the replica_axis flag I made there, to keep everything as is in the single replica models (i.e. not have an extra replica axis of length 1 there) so nothing there has to change.
  • It is producing very different results, because the initialization is different. Keeping it the same would be very ugly, but I can do it temporarily just to verify that that's the only source of differences.
  • Fix test_fit

Edit: the changes of this PR changed the layout of the weights and thus weights saved for earlier versions of the code need to be restructured so they remain equivalent. The following script (comment: https://github.com/NNPDF/nnpdf/pull/1881#issuecomment-1885018093) achieves exactly that:

test script 
import numpy as np
import h5py

def reformat_h5(i_replica):
    old_file = h5py.File(f"weights_{i_replica}_master.h5", 'r')
    new_file = h5py.File(f"weights_{i_replica}.h5", 'r+')

    def replace_numbers(name, obj):
        name_new_to_old_rules = {
                "NNs": f"NN_{i_replica - 1}",
                "preprocessing_factor": f"preprocessing_factor_{i_replica - 1}",
                "dense_3": "dense",
                "dense_4": "dense_1",
                "dense_5": "dense_2",
            }
        if isinstance(obj, h5py.Dataset):
            old_name = name
            for new, old in name_new_to_old_rules.items():
                old_name = old_name.replace(new, old)

            print(f"Replacing {name} with {old_name}")
            data_to_put_in = old_file[old_name][...]
            if "preprocessing" in name:
                data_to_put_in = np.expand_dims(data_to_put_in, axis=0)

            obj[...] = data_to_put_in

    new_file.visititems(replace_numbers)

reformat_h5(1)
reformat_h5(2)

Merge request reports