Tensorboard and Keras Images with ImageDataGenerator

Question

I have a problem while using TensorFlow, Keras and TensorBoard.

I wrote an image classification network. I use ImageDataGenerator from Keras to load the images during the fitting:

test_image_data = test_image_generator.flow_from_directory("images/test_set/buildings",
                                                       target_size=(64, 64),
                                                       batch_size=32,
                                                       class_mode='binary')

tensorboard = TensorBoard(log_dir='./log/{}'.format(NAME), embeddings_freq=1, embeddings_layer_names=['features'], embeddings_data=test_image_data)

model.fit_generator(train_image_data, validation_data=test_image_data, validation_steps=800,
                    steps_per_epoch=8000, epochs=10, callbacks=[tensorboard])

So I want to look up in TensorBoard what the CNN does with the images.

Thanks, Marvin.

Answer 1

I couldn't figure out, how to do this with the tensorboard callback in keras as well. Therefore I wrote my own keras callback. Here is a short code snippet, you can modify it easily

class ImageCallback(Callback):
    def __init__(self,logdir, data, n_images):
        self.logdir = logdir
        self.data = data
        self.n_images = n_images
    def set_model(self,model):
        self.model = model
        self.sess = K.get_session()
        self.writer = tf.summary.FileWriter(logdir=self.logdir)
    def on_epoch_end(self,epoch,logs={}):
        output_images = tf.cast(self.model.call(self.data),dtype=tf.float32)
        output_images *= 255
        with tf.name_scope("predictions"):
            tf.summary.image(name="output",tensor = output_iamges, max_outputs=self.n_images)
        self.writer.add_summary(self.sess.run(self.summary), epoch+1)
    def on_train_begin(self,logs=None):
        with tf.name_scope("images"):
            input_images = tf.summary.image(name="input", tensor=self.data, max_outputs=self.n_images)
        self.writer.add_summary(self.sess.run(input_images))
        output_images = tf.cast(self.model.call(self.data),dtype=tf.float32)
        output_images *= 255
        with tf.name_scope("predictions"):
            tf.summary.image(name="output",tensor = output_images, max_outputs=self.n_images)
       self.summary = tf.summary.merge_all(scope='predictions')
       self.writer.add_summary(self.sess.run(self.summary),0)

Then you can pass this callback when calling model.fit() . I hope that helps. Let me know, if there is a better way of doing it.