Force @tf.function in all operations and layer calls

Created by: scarlehoff

Compiling what can be compiled results in a better performance.

This PR also separates the convolution as a separate operation and uses tensors at some points where before I was using numpy arrays. As a result the memory consumption goes down. The reason being these numpy arrays would force a different compilation per operation whereas now it is a single operation that takes some placeholder that's filled in place.