How to troubleshoot custom Keras layers

Question

I'm creating a model using Keras (I'm a beginner) and wrote a lambda function to randomly choose whether or not to flip the initial input layer.

This is the snippet that attempts to do so:

input_global = Input(shape=(2001,1))
flipped = Lambda(lambda x: keras.backend.reverse(x, axes=1) if np.random.random() < 0.5 else x, output_shape=(input_global.shape[1], input_global.shape[2]))(input_global)

My model compiles, yet logging messages like print("Hello, world") in the middle of the definition of the model only result in "Hello world" being logged once- not every time data goes into the model during training.

How do I know that my function did what I intended it to do?

Answer 1

When building a Keras model, you are building a graph of operations, and whenever you run input through your model, you are running your input through that graph of operations. In other words, you build a model once, and then you can run input through it as many times as you'd like.

Non-Keras functions will not be part of the graph, including operations like print or np.random.random() . You will need to use the Keras equivalents.

For print , use the function keras.backend.print_tensor .

For np.random.random , use the function keras.backend.random_uniform .

I think you did not intend to do this, but because you are using np.random.random() in your custom layer, that operation will be executed only when you build the model . It will not be run whenever you run data through your model. In other words, the layer will either only reverse the inputs, or it will only return the inputs. To get the random behavior I think you want, you need to use a Keras function (ie, keras.backend.random_uniform ). Using that Keras function will generate a random number every time you run your model.

To belabor the point, np.random.random() will be run once when you build the model, and in essence it will determine at build time what that layer will do (ie, it will either only reverse the input, or it will only return the input unchanged). keras.backend.random_uniform() , on the other hand, will not generate a random number when you build the model, but instead it adds a random_uniform operation to the model, so every time data goes through the model, a random number will be drawn from a uniform distribution at that point.