修改 TensorFlow 中恢復的 CNN model 的權重和偏差

Question

我最近開始使用 TensorFlow (TF)，遇到一個問題需要一些幫助。 基本上，我恢復了一個預訓練的 model，我需要在重新測試其准確性之前修改其中一層的權重和偏差。 現在，我的問題如下：如何使用 TF 中的assign方法更改權重和偏差？ 在 TF 中甚至可以修改恢復模型的權重嗎？

這是我的代碼：

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data # Imports the MINST dataset

# Data Set:
# ---------
mnist = input_data.read_data_sets("/home/frr/MNIST_data", one_hot=True)# An object where data is stored

ImVecDim = 784# The number of elements in a an image vector (flattening a 28x28 2D image)
NumOfClasses = 10

g = tf.get_default_graph()

with tf.Session() as sess:
  LoadMod = tf.train.import_meta_graph('simple_mnist.ckpt.meta')  # This object loads the model
  LoadMod.restore(sess, tf.train.latest_checkpoint('./'))# Loading weights and biases and other stuff to the model

  # ( Here I'd like to modify the weights and biases of layer 1, set them to one for example, before I go ahead and test the accuracy ) #

  # Testing the acuracy of the model:
  X = g.get_tensor_by_name('ImageIn:0')
  Y = g.get_tensor_by_name('LabelIn:0')
  KP = g.get_tensor_by_name('KeepProb:0')
  Accuracy = g.get_tensor_by_name('NetAccuracy:0')
  feed_dict = { X: mnist.test.images[:256], Y: mnist.test.labels[:256], KP: 1.0 }
  print( 'Model Accuracy = ' )
  print( sess.run( Accuracy, feed_dict ) )

Answer 1

除了現有答案之外，還可以通過tf.assign function 執行張量更新。

v1 = sess.graph.get_tensor_by_name('v1:0')
print(sess.run(v1))   # 1.0
sess.run(tf.assign(v1, v1 + 1))
print(sess.run(v1))   # 2.0

Answer 2

感謝所有回復的人。 我只想把這些碎片拼湊起來。 這是幫助我完成我想要的代碼：

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data # Imports the MINST dataset

# Data Set:
# ---------
mnist = input_data.read_data_sets("/home/frr/MNIST_data", one_hot=True)# An object where data is stored

ImVecDim = 784# The number of elements in a an image vector (flattening a 28x28 2D image)
NumOfClasses = 10

g = tf.get_default_graph()

with tf.Session() as sess:
   LoadMod = tf.train.import_meta_graph('simple_mnist.ckpt.meta')  # This object loads the model
   LoadMod.restore(sess, tf.train.latest_checkpoint('./'))# Loading weights and biases and other stuff to the model

   wc1 = g.get_tensor_by_name('wc1:0')
   sess.run( tf.assign( wc1,tf.multiply(wc1,0) ) )# Setting the values of the variable 'wc1' in the model to zero.

   # Testing the acuracy of the model:
   X = g.get_tensor_by_name('ImageIn:0')
   Y = g.get_tensor_by_name('LabelIn:0')
   KP = g.get_tensor_by_name('KeepProb:0')
   Accuracy = g.get_tensor_by_name('NetAccuracy:0')
   feed_dict = { X: mnist.test.images[:256], Y: mnist.test.labels[:256], KP: 1.0 }
   print( 'Model Accuracy = ' )
   print( sess.run( Accuracy, feed_dict ) )

Answer 3

是的，有可能。 加載元圖后，您的權重和偏差已經加載。 您需要找到它們的名稱（請參閱list_variables函數），然后將它們分配給一個 Python 變量。

為此， tf.get_variable與變量名稱一起使用。 您可能必須在變量 scope 上設置reuse=True 。有關重用變量的更多詳細信息，請參閱此答案。

一旦將它們作為weights變量，就可以調用sess.run(weights.assign(...)) 。

Answer 4

使用與 OP 不同的示例對 Tensorflow 2.4 進行更新。

# Step 0 - Init
model        = # some tf.keras.Model
model_folder = # path to model files
    
ckpt_obj = tf.train.Checkpoint(model=model)
ckpt_obj.restore(save_path=tf.train.latest_checkpoint(str(model_folder))).expect_partial()

# Step 1 - Loop over all layers
for layer in model.layers:

    # Step 2 - Loop over submodules of a layer
    for submodule in layer.submodules:

        # Step 3 - Find a particular type of submodule (alternative use submodule.name=='SomeName')
        if type(submodule) == tfp.layers.Convolution3DFlipout: # kernel=N(loc,scale) --> N=Normal distro
            
            # Step 4 - Extract numpy weights using .get_weights()
            ## Note: Different compared to submodule.weights which returns a tensor that shall also have a name e.g. wc1:0
            weights = submodule.get_weights() # [scale, rho, bias] --> kernel=N(loc,scale=tfp.bijectors.Softplus(rho)) --> output=input*kernel + bias
            
            # Step 5 - Set weights as a new numpy array of your choice
            weights[1] = np.full(weights[1].shape, -np.inf) 

            # Step 6 - Update weights
            submodule.set_weights(weights)


input = tf.random.normal((1,100,100,100,1)) # 3D input with batch=1, channels=1
_ = model(input)