Skip to content

Set tensorflow-eigen in the conda recipe

Emanuele Roberto Nocera requested to merge back_to_eigen into master

Created by: scarlehoff

Yesterday I spent several hours after the phone call trying to figure out how the conda is build differently so that it creates memory leaks. Was not able to really figure out anything useful and also building TF can take hours to build even in the big machines here in Milan so debugging the community builds is out of the question.

Furthermore I can see a growth of memory when using 2.2 with respect to 2.1 even for simple (non n3fit) models so I believe this is an actual problem with the packages. Might be solved for 2.3, might not, it was ok for 2.1 now it is bad for 2.2. We saw already similar problems with 2.0 I believe.

There was a quick solution to the problem, and it was having the environmental variable:

export TF_DISABLE_MKL=1

i..e, clearly there is a problem with the mkl version of tensorflow.

so my suggestions (included in this PR) are:

  • Set the conda tensorflow package to eigen, as it seems to work well for all versions (2.0, 2.1, 2.2) so seems to be stable enough.* This allows us to have a self-contained conda package (which is good and make people happy) The main advantage that I found for mkl was a slightly better performance in many-cores machine, if you are looking for that kind of fine-grained performance you might as well go through whatever loops you might find using pip. By doing this instead of disabling MKL inside TF the user can decide whether to use MKL or not (because maybe for 2.3 or 2.4 or who knows works fine).

  • Write somewhere in the documentation that the best version is expected to be the one coming from pip

  • Forget about this problem 4ver

Merge request reports

Loading