Skip to content

Generate replicas when fitting models in parallel

Emanuele Roberto Nocera requested to merge n3fit_replica_range_parallel into use_n3pdf_interface_with_members

Created by: scarlehoff

As the title says, this enables using genrep=True when fitting models in parallel.

Since we already have the mechanism for generating more than one replica (with the replica range) I've changed the parallel_models: int to parallel_models: bool and the replicas are given with the replica range. In this way running in parallel is the same as running sequentially in terms of seeds and data.

The process is:

  1. Generate all replicas before starting to fit (just as it was done before when using -r)
  2. If parallel_models: True then all replicas are exactly the same (same trvl, same fktable, same invcovmat)
  3. Create an output which is a stack of all the replicas (so the output is (n_replicas, n_data) instead of (1, n_data))
  4. Fit normally (I left the output in PR #1153 zs (1, n_data) so that this could be implemented easily and it worked out! :) )

For reviewers: this is implemented in the first commit in n3fit/src/n3fit/model_trainer.py . Most other changes are changes of variable names to use the plural form (like seed -> seeds) plus some typos I found (quite a few in checks.py)

Working on top of PR #1251 (which is smaller than this PR and the changes are even less impactful so please go to that one first)

A draft for now since I'd like to know how reproducible results are between running sequential / parallel / one by one and add that to the docs.

Merge request reports