I am confused to add new class for a pre-trained model, what i have done until now restore the pre-trained checkpoint and create a matrix of size m * C+1 and a vector of length C+1, then initialize the first C rows/elements of these from the existing weights and freeze the previous layer by training just the FC layer in the Optimizer.minimize(). However when i run the code, i got this error :
Traceback (most recent call last):
File "/home/tensorflow/tensorflow/models/image/mnist/new_dataset/Nets.py", line 482, in <module>
new_op_w = optimizer_new.minimize(loss, var_list = resize_var_w)
File "/home/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/optimizer.py", line 279, in minimize
grad_loss=grad_loss)
File "/home/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/optimizer.py", line 337, in compute_gradients
processors = [_get_processor(v) for v in var_list]
File "/home/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 502, in __iter__
raise TypeError("'Tensor' object is not iterable.")
TypeError: 'Tensor' object is not iterable.
and that's the code :
with tf.Session(graph=graph) as sess:
if os.path.isfile(ckpt):
aver.restore(sess, 'path_to_checkpoint.ckpt')
w_b_new = {
'weight_4': tf.Variable(tf.random_normal([num_hidden, 1], stddev=0.1), name = 'weight_4'),
'bias_4' : tf.Variable(tf.constant(1.0, shape=[1]), name = 'bias_4'),}
change_1 = tf.unstack(w_b_not['weight_4'])
change_2 = tf.unstack(w_b_not['bias_4'])
change_3 = tf.unstack(w_b_new['weight_4'])
change_4 = tf.unstack(w_b_new['bias_4'])
changestep1 = []
for i in range(len(change_1)):
changestep1.append(tf.unstack(change_1[i]))
changestep3 = []
for i in range(len(change_3)):
changestep3.append(tf.unstack(change_3[i]))
for j in range(len(changestep3[i])):
changestep1[i].append(changestep3[i][j])
changestep1[i] = tf.stack(changestep1[i])
final1 = tf.stack(changestep1)
resize_var_w = tf.assign(w_b_not['weight_4'], final1, validate_shape=False)
final2 = tf.concat([w_b_not['bias_4'] , w_b_new['bias_4']], axis=0)
resize_var = tf.assign(w_b_not['bias_4'], final2, validate_shape=False)
optimizer_new = tf.train.GradientDescentOptimizer(0.01)
new_op_w = optimizer_new.minimize(loss, var_list = resize_var_w)
new_op_b = optimizer_new.minimize(loss, var_list = resize_var)
for step in range(num_steps,num_steps + num_train_steps):
offset = (step * batch_size) % (train_labels.shape[0] - batch_size)
batch_data = train_dataset[offset:(offset + batch_size), :, :, :]
batch_labels = train_labels[offset:(offset + batch_size), :]
feed_dict = {tf_train_dataset : batch_data, tf_train_labels : batch_labels , keep_prob:0.5}
_,_, l, predictions = sess.run([new_op_w,new_op_b, loss, train_prediction ], feed_dict=feed_dict)
if (step % 50 == 0):
print('%d\t%f\t%.1f%%\t%.1f%%' % (step, l, accuracy(predictions, batch_labels), accuracy(valid_prediction.eval(), valid_labels)))
print('Test accuracy: %.1f%%' % accuracy(test_prediction.eval() , test_labels))
save_path_w_b = saver.save(sess, "path_checkpoint.ckpt")
print("Model saved in file: %s" % save_path_w_b)
According to the TensorFlow docs on GradientDescentOptimizer's minimize method , "var_list" must be a list of Variable objects. According to your code, resize_var_w
is a single tensor.
EDIT To be more specific:
If you give the optimizer var_list
, as the name implies, this must be a list of variables. During backprop the optimizer will loop var_list
and only update the variables in the list, as opposed to all trainable variables in the graph. A single variable is not iterable.
If you only want to update a single Tensor, you can simply try:
resize_var_w = [tf.assign(w_b_not['weight_4'], final1, validate_shape=False)]
I did not test, but should work.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.