n3fit memory problem TF 2.0

Created by: wilsonmr

I believe I am still getting a memory problem with TF 2.0

version:

tensorflow                2.0.0           mkl_py37h66b46cc_0

the closure branch was before updating to use new TF, and so I was still using the work around in #522 (closed)

I did notice in the log this little block of messages - not sure if relevant?

[WARNING]: From /exports/csce/eddie/ph/groups/nnpdf/Users/wilsonmr/Programs/anaconda/envs/tf2/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
[INFO]: Generating the Model
[WARNING]: OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
2019-11-18 10:36:55.213546: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
2019-11-18 10:36:55.222552: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2400150000 Hz
2019-11-18 10:36:55.223135: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5611c2908000 executing computations on platform Host. Devices:
2019-11-18 10:36:55.223167: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version

in particular the

OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs

I guess this is set by the Edinburgh cluster? Or is it us?

After #605 I don't believe I can use the conda-forge version of TF, so I'm in a little bit of a pickle.