[英]Modifying the weights and biases of a restored CNN model in TensorFlow
I have recently started using TensorFlow (TF), and I have come across a problem that I need some help with.我最近开始使用 TensorFlow (TF),遇到一个问题需要一些帮助。 Basically, I've restored a pre-trained model, and I need to modify the weights and biases of one of its layers before I retest its accuracy.
基本上,我恢复了一个预训练的 model,我需要在重新测试其准确性之前修改其中一层的权重和偏差。 Now, my problem is the following: how can I change the weights and biases using the
assign
method in TF?现在,我的问题如下:如何使用 TF 中的
assign
方法更改权重和偏差? Is modifying the weights of a restored modeled even possible in TF?在 TF 中甚至可以修改恢复模型的权重吗?
Here is my code:这是我的代码:
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data # Imports the MINST dataset
# Data Set:
# ---------
mnist = input_data.read_data_sets("/home/frr/MNIST_data", one_hot=True)# An object where data is stored
ImVecDim = 784# The number of elements in a an image vector (flattening a 28x28 2D image)
NumOfClasses = 10
g = tf.get_default_graph()
with tf.Session() as sess:
LoadMod = tf.train.import_meta_graph('simple_mnist.ckpt.meta') # This object loads the model
LoadMod.restore(sess, tf.train.latest_checkpoint('./'))# Loading weights and biases and other stuff to the model
# ( Here I'd like to modify the weights and biases of layer 1, set them to one for example, before I go ahead and test the accuracy ) #
# Testing the acuracy of the model:
X = g.get_tensor_by_name('ImageIn:0')
Y = g.get_tensor_by_name('LabelIn:0')
KP = g.get_tensor_by_name('KeepProb:0')
Accuracy = g.get_tensor_by_name('NetAccuracy:0')
feed_dict = { X: mnist.test.images[:256], Y: mnist.test.labels[:256], KP: 1.0 }
print( 'Model Accuracy = ' )
print( sess.run( Accuracy, feed_dict ) )
Thanks for everyone who responded.感谢所有回复的人。 I'd just like to put the pieces together.
我只想把这些碎片拼凑起来。 This is the code the helped me accomplish what I want:
这是帮助我完成我想要的代码:
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data # Imports the MINST dataset
# Data Set:
# ---------
mnist = input_data.read_data_sets("/home/frr/MNIST_data", one_hot=True)# An object where data is stored
ImVecDim = 784# The number of elements in a an image vector (flattening a 28x28 2D image)
NumOfClasses = 10
g = tf.get_default_graph()
with tf.Session() as sess:
LoadMod = tf.train.import_meta_graph('simple_mnist.ckpt.meta') # This object loads the model
LoadMod.restore(sess, tf.train.latest_checkpoint('./'))# Loading weights and biases and other stuff to the model
wc1 = g.get_tensor_by_name('wc1:0')
sess.run( tf.assign( wc1,tf.multiply(wc1,0) ) )# Setting the values of the variable 'wc1' in the model to zero.
# Testing the acuracy of the model:
X = g.get_tensor_by_name('ImageIn:0')
Y = g.get_tensor_by_name('LabelIn:0')
KP = g.get_tensor_by_name('KeepProb:0')
Accuracy = g.get_tensor_by_name('NetAccuracy:0')
feed_dict = { X: mnist.test.images[:256], Y: mnist.test.labels[:256], KP: 1.0 }
print( 'Model Accuracy = ' )
print( sess.run( Accuracy, feed_dict ) )
Yes it is possible.是的,有可能。 Your weights and biases are already loaded after you loaded the meta graph.
加载元图后,您的权重和偏差已经加载。 You need to find their names (see the list_variables function) and then assign them to a Python variable.
您需要找到它们的名称(请参阅list_variables函数),然后将它们分配给一个 Python 变量。
For that, use tf.get_variable
with the variable name.为此,
tf.get_variable
与变量名称一起使用。 You might have to set reuse=True
on your variable scope. See this answer for more detail on reusing variables.您可能必须在变量 scope 上设置
reuse=True
。有关重用变量的更多详细信息,请参阅此答案。
Once you have them as a weights
variable, you can call sess.run(weights.assign(...))
.一旦将它们作为
weights
变量,就可以调用sess.run(weights.assign(...))
。
An update to this for Tensorflow 2.4 using a different example than OP's.使用与 OP 不同的示例对 Tensorflow 2.4 进行更新。
# Step 0 - Init
model = # some tf.keras.Model
model_folder = # path to model files
ckpt_obj = tf.train.Checkpoint(model=model)
ckpt_obj.restore(save_path=tf.train.latest_checkpoint(str(model_folder))).expect_partial()
# Step 1 - Loop over all layers
for layer in model.layers:
# Step 2 - Loop over submodules of a layer
for submodule in layer.submodules:
# Step 3 - Find a particular type of submodule (alternative use submodule.name=='SomeName')
if type(submodule) == tfp.layers.Convolution3DFlipout: # kernel=N(loc,scale) --> N=Normal distro
# Step 4 - Extract numpy weights using .get_weights()
## Note: Different compared to submodule.weights which returns a tensor that shall also have a name e.g. wc1:0
weights = submodule.get_weights() # [scale, rho, bias] --> kernel=N(loc,scale=tfp.bijectors.Softplus(rho)) --> output=input*kernel + bias
# Step 5 - Set weights as a new numpy array of your choice
weights[1] = np.full(weights[1].shape, -np.inf)
# Step 6 - Update weights
submodule.set_weights(weights)
input = tf.random.normal((1,100,100,100,1)) # 3D input with batch=1, channels=1
_ = model(input)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.