TensorFlow: Adding a small noise to pre-trained weights

Question

I have finished training the stack of 3 LSTMs and now I want to investigate how sensitive is each LSTM layer to small perturbations of its weights. I would like to load the model from checkpoints, add a small value to weights of a certain layer and record performance losses/gains. I wonder what is the simplest way to do this?

Answer 1

import tensorflow as tf

# define model
...

# load checkpoint
...

# assemble the list of weights to add noise
list_of_weights = [ ... ] 


def add_random_noise(w, mean=0.0, stddev=1.0):
    variables_shape = tf.shape(w)
    noise = tf.random_normal(
        variables_shape,
        mean=mean,
        stddev=stddev,
        dtype=tf.float32,
    )
    return tf.assign_add(w, noise)


sess = tf.Session()
for w in list_of_weights:
    sess.run(add_random_noise(w))

# continue experiments
...