How do I debug keras model

Question

I am going through tutorial for handwritten text recognition. And to do hand written digit recognition the author has constructed a Keras model as follows:

# # Creating CNN model

input_shape = (28,28,1)
number_of_classes = 10

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=input_shape))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))

model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))

model.add(Dropout(0.5))
model.add(Dense(number_of_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),metrics=['accuracy'])

model.summary()

history = model.fit(X_train, y_train,epochs=5, shuffle=True,
                    batch_size = 200,validation_data= (X_test, y_test))


model.save('digit_classifier2.h5')

Source ( here )

I am very confused that on how has the author choose these layers. I know how Conv2D works by applying filters to an image, I know what is activation function . In short I have a rough understanding of what each term means.

What I am finding it difficult is how do I know what is happening in each step of this code? For example lets take this python code:

values_List=[11,34,43]
for index, num in enumerate(values_List):
    print(index,num)

1. I know that line 1 initializes a list named values_List
2. Line 2 iterates through this list
3. Line 3 prints output as (index of a number, number)

This python code is easy to understand and debug. But I am confused that if there is any error inside the keras layers. How do I proceed to debug this Keras code? How do I see output on each step inside the Keras code?

Answer 1

In short, you can't easily debug in Keras cause it is a high-level API made for the faster and easier implementations of Neural network architecture using pre-defined layers and functions there is less chance of error inside these layers or function cause it is well tested.

If you want to more fine-grained control on you you need to implement in Low-level API like Tensorflow v1 or use tf.GradientTape with tf-keras in TensorFlow v2 to see gradients at each step.

You can also try Tensorwatch by Microsoft for a deeper understanding of your model - https://github.com/microsoft/tensorwatch