简体   繁体   中英

TF sess.run() creates new objects of my CustomOp when I include the optimizer

Following the example from here [ https://www.tensorflow.org/extend/adding_an_op] , I created my own operator, let's call it MyConv .

Now, if I make something like this:

# placeholders for inputs and labels
input = tf.placeholder(...)
label = tf.placeholder(...)

# MyConv operator on inputs... returns 1D tensor
conv_out = my_module.my_conv(input, ...)
# final classifier 
output = tf.layers.dense(inputs=conv_out, units=10)

# eveything we need for backprop
probs = tf.losses.sparse_softmax_cross_entropy(logits=output, labels=label)
loss = tf.reduce_mean(probs)
adam_opt = tf.train.AdamOptimizer(learning_rate=0.01)
train_op = adam_opt.minimize(loss)

And run the model N-1 times with a simple input forwarding, and only the Nth time with the backprop:

num_of_chunks = 100

init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    for c in range(num_of_chunks):
        single_input = np.array(...)
        # for num_of_chunks-1 times, just push the single_input through the network
        if c != num_of_chunks - 1:
            result = sess.run([output], feed_dict={input:single_input})
        # only the last time, do the backprop and run the optimizer
        else:
            result, loss_out, train_out = sess.run([output, loss, train_op], feed_dict={input:single_input, label:np.array(...)})

An interesting thing occurs. TF sess.run() method will instantiate one object of MyConv class and use it num_of_chunks-1 times, but for the last time (when it executes this else path) it will create a new object of class MyConv and use that new object (MyConv() constructor has been invoked one more time). It looks like, just because in the else path I have introduced the backprop, sess.run() creates new objects of MyConv operator, and that's not really a behavior I have expected (I store some information there as a private class member, and I would really like to keep one MyConv object while running the session).

So, the question is: do you know why would sess.run() method create a new object of an operator just because I have switched from 'only-forward mode' to 'forward and backprop mode'?

Also, one note. If I remove this if-else statement and let the backprop to be executed for every chunk, all is good, only one object of MyConv is instantiated and used.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM