n3fit - Refactor of hyperopt

Created by: scarlehoff

It is basically a refactoring of the hyperopt code so it includes more documentation and it is easier to follow.

How to

The first step is to prepare a runcard with a hyperscan dictionary in it. As an example you can see the one in the runcards folder of n3fit: Basic_runcard.yml.

In its current form the hyperoptimization is activated by the flag --hyperopt at run time. It accepts as an argument a number, which is basically the number of trials to run. So, once you have your runcard prepare all you have to do is:

n3fit my_hyper_runcard.yml [replica_number] --hyperopt [number of trials] Each trial consists on a different configuration of the parameter, following the scan defined in hyperscan.

Once the hyperopt scan has finished (it will take slightly less than time_of_a_single_fit * nuumber_of_trials) you will find a file in the nnfit directory:

my_hyper_runcard/nnfit/replica_1/trials.json

With this you can use the n3Hyperplot script to generate the plots:

n3Hyperplot my_hyper_runcard

Will generate a plot per replica. If you want to join all replicas as if it were one single longer run you can do

n3Hyperplot my_hyper_runcard --combine

If you want to play a bit with the results, maybe remove anything that uses Adadelta as the optimizer and has less than 500 epochs:

n3Hyperplot my_hyper_runcard --combine --filter "optimizer=Adadelta" "epochs>500"

The filter option will accept the operators !=, =, <, >

Note: I generally do the hyperscan with the replicas turned off, but send several runs with a different replica number, this way I get a few .json files with statistically different runs (all the seeding in n3fit depends on the replica number)