Skip to content

Add more regression tests

Emanuele Roberto Nocera requested to merge more_restrictive_regressions into master

Created by: scarlehoff

This would've caught several errors that found their way into master (and that, for instante, delayed the qed paper!)

Of course, this is going to introduce a lot of false positives, but I think the trade-off will be worth it (and if not, we can always roll it back or make the check a bit more tolerant).

This PR:

  1. Adds check on the final .exportgrid produced by the regression tests inside the n3fit tests.
  2. Adds bunch of extra regression tests for fits as a separate run workflow
  3. Separates integrability and arc_lenghts into their own category so they are checked with a greater tolerance. integrability is too close to 1e-8 and asking for too much precision generates problem. arc_length on the other hand involves an integration using scipy (I believe this is what you are seeing in your other PR @APJansen). Maybe there's some environmental variable that makes it more robust...

Note that this separate workflow at the moment only deals with fits. The plan is to also add validphys runcards in the future.

Merge request reports