How to print value during Training in Tensorflow 1.x

Question

I have a training setup in tensorflow 1.x. I want to print some value in tensorflow.python.ops files during execution. So I have tried a lot of ways to print this out, but all is failed Creating a session can't be applied in this case, because the ops have already being executed in the training session.

I have tried to enable eager_execution at first, and use tf.print , but nothing show out.

Answer 1

you will have to use a custom callback to achieve printing out of data during training. Not sure what type of data you wish to print but here is the code for a custom callback that will print out data on the screen and also write that data to a file. In this case as a demo I am writing out the validation loss for the epoch. The data printed on the screen is actually the data for the previous epoch and is printed at the start of the next epoch. If you try to print to the screen at the end of an epoch it messes up the printout that tensorflow generates at the end of an epoch. Below is the code for the custom callback

class p_info(keras.callbacks.Callback):
    def __init__(self,print_path):
        super(p_info, self).__init__()
        self.print_path=print_path  # path to file you want to print to
        msg=''
    def on_epoch_begin(self, epoch, logs=None): # just used to print data from previous epoch
        if epoch != 0:
            print (p_info.msg) # print a message generated at end of previous epoch during training           
        return          
    def on_epoch_end(self, epoch, logs=None):
        v_loss=logs.get('val_loss')  # get the validation loss for this epoch        
        p_info.msg=' value loss is = '+ str(t_loss) + ' for epoch ' +str(epoch) + '\n' # test you want to store in the file
        file1 = open(self.print_path, 'a') # open file in append mode
        file1.write(p_info.msg)
        file1.close()
        return

in your code before model.fit include the code below

file_path=r'c:\my_file.txt'  #  insert here the full path of the file you want to store data in
callbacks=[p_info(file_path)]

in model.fit include callbacks=callbacks. I do not know what kind of data you want to generate at the end of an epoch. You can use logs.get(a metric) to get the value of a metric so in your case you may want to define a custom metric that contains the information you want. Documentation for creating custom metrics is here.