Following the example from here [ https://www.tensorflow.org/extend/adding_an_op] , I created my own operator, let's call it MyConv
.
Now, if I make something like this:
# placeholders for inputs and labels
input = tf.placeholder(...)
label = tf.placeholder(...)
# MyConv operator on inputs... returns 1D tensor
conv_out = my_module.my_conv(input, ...)
# final classifier
output = tf.layers.dense(inputs=conv_out, units=10)
# eveything we need for backprop
probs = tf.losses.sparse_softmax_cross_entropy(logits=output, labels=label)
loss = tf.reduce_mean(probs)
adam_opt = tf.train.AdamOptimizer(learning_rate=0.01)
train_op = adam_opt.minimize(loss)
And run the model N-1 times with a simple input forwarding, and only the Nth time with the backprop:
num_of_chunks = 100
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
for c in range(num_of_chunks):
single_input = np.array(...)
# for num_of_chunks-1 times, just push the single_input through the network
if c != num_of_chunks - 1:
result = sess.run([output], feed_dict={input:single_input})
# only the last time, do the backprop and run the optimizer
else:
result, loss_out, train_out = sess.run([output, loss, train_op], feed_dict={input:single_input, label:np.array(...)})
An interesting thing occurs. TF sess.run()
method will instantiate one object of MyConv
class and use it num_of_chunks-1
times, but for the last time (when it executes this else
path) it will create a new object of class MyConv
and use that new object (MyConv() constructor has been invoked one more time). It looks like, just because in the else
path I have introduced the backprop, sess.run()
creates new objects of MyConv
operator, and that's not really a behavior I have expected (I store some information there as a private class member, and I would really like to keep one MyConv
object while running the session).
So, the question is: do you know why would sess.run() method create a new object of an operator just because I have switched from 'only-forward mode' to 'forward and backprop mode'?
Also, one note. If I remove this if-else
statement and let the backprop to be executed for every chunk, all is good, only one object of MyConv
is instantiated and used.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.