.pb模型中的Tensorflow Adam优化器

Question

I have developed a TensorFlow code that use Adam optimizer, then saved the graph and export the .pb model and correctly loaded it, my problem is when i feed it with new input image i don't get the same result compared to the result given by this code:

from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img, array_to_img 

import cv2
import tensorflow as tf
import numpy
import numpy as np




def get_image2(imgSrc):
    img = load_img(imgSrc, True)  # this is a PIL image
    x = img_to_array(img)  # this is a Numpy array with shape (3, 150, 150
    #x = x.reshape((1,) + x.shape)
    x = x.astype(float)
    x *= 1./255.
    #x = cv2.resize(x,(512,512))
    return x


def sobel2(image):


    # Shape = height x width.
    #image = tf.placeholder(tf.float32, shape=[None, None])

    # Shape = 1 x height x width x 1.
    image_resized = image#tf.expand_dims(image, 0)

    Gx = tf.nn.conv2d(image_resized, sobel_x_filter, strides=[1, 1, 1, 1], padding='SAME')
    Gy = tf.nn.conv2d(image_resized, sobel_y_filter,strides=[1, 1, 1, 1], padding='SAME')

    #grad = tf.sqrt(tf.add(tf.pow(Gx,2),tf.pow(Gy,2)))
    #grad = tf.pow(Gx,2) + tf.pow(Gy,2)
    #grad = tf.truediv(grad,3.)

    #grad = tf.reshape(grad, img_shape)

    return Gx, Gy

image = get_image2('1.jpg')

img_shape = image.shape
print img_shape
img_h, img_w,_= img_shape

sobel_x = tf.constant([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], tf.float32)
sobel_x_filter = tf.reshape(sobel_x, [3, 3, 1, 1])
sobel_y_filter = tf.transpose(sobel_x_filter, [1, 0, 2, 3])

input_img  = tf.placeholder(tf.float32, shape=[1,img_shape[0],img_shape[1],img_shape[2]], name="input_img") 
#input_img  = tf.placeholder(tf.float32, [1, 512, 512, 1], name="input_img")


gain   = tf.Variable(tf.constant(1, dtype=tf.float32, shape=[1,img_shape[0],img_shape[1],img_shape[2]]), name="gain")
offset = tf.Variable(tf.constant(0, dtype=tf.float32, shape=[1,img_shape[0],img_shape[1],img_shape[2]]), name="offset")

enhanced_img = tf.add(tf.multiply(input_img, gain), offset, name = "enahnced")


#----------------------------------------------------------
# COST
#----------------------------------------------------------

input_img_deriv_x, input_img_deriv_y    = sobel2(input_img)

enhanced_img_deriv_x, enhanced_img_deriv_y = sobel2(enhanced_img)

white_img = tf.constant(1, dtype=tf.float32, shape=[1,img_shape[0],img_shape[1],img_shape[2]])

image_pixels_count = img_h * img_w

white_cost = tf.reduce_sum(tf.pow(enhanced_img - white_img, 2))
sobel_cost = tf.reduce_sum(tf.pow(enhanced_img_deriv_x - input_img_deriv_x, 2) + 
                            tf.pow(enhanced_img_deriv_y - input_img_deriv_y,2))
cost = tf.add(white_cost, tf.multiply(0.2, sobel_cost), name = "cost") # + tf.reduce_sum(gain - 1) + tf.reduce_sum(offset)


#----------------------------------------------------------
# TRAIN
#----------------------------------------------------------

# Parameters
learning_rate = 0.0001
training_epochs = 100
display_step = 5

optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)

# Initialize the variables (i.e. assign their default value)
init = tf.global_variables_initializer()

saver = tf.train.Saver()
image  = image.reshape([-1,img_shape[0],img_shape[1],img_shape[2]])
#print image.shape

#print image.shape
feed = {input_img: image }



# Start training
with tf.Session() as sess:
    #Run the initializer
    print(sess.run(init))

    # Fit all training data
    for epoch in range(training_epochs):
        sess.run([optimizer, cost], feed_dict = feed)
        print(tf.reduce_sum(offset.eval()).eval())


        if (epoch+1) % display_step == 0:
            gen_img = sess.run(enhanced_img, feed_dict = feed)
        gen_img  = np.squeeze(gen_img, axis=0)
            print(gen_img.shape)
            gen_img *= 255
            cv2.imwrite("result/output_2_{0}.png".format(epoch), gen_img)

I noticed that when i save the graph the optimizer state is also saved, so when i load the model and feed it with new image he will produce a false result since he will use the saved value related to the image i have used when i saved it.

How i can make the model run the optimizer for new images without using the saved parameters from previous input.

Answer 1

Do you understand how the optimizer actually works ?

The goal of the optimizer is to update the model weights with respect to the gradient. Concerning Adam it has two inner variables which are updated during training , it's part of the adam algorithm. So this behavior is perfectly normal . If you want to "reset" adam variables, it's perfectly doable, however I highly doubt that it's what you want to do... Very rare situations require you to do this. Btw. if you reset adam state, you will break the whole logic of the optimizer.

If you try to evaluate a new image at inference time, the optimizer should not be run, and thus your model output should not be impacted by Adam or any other optimizer.

If you try to continue the training from a preivously saved checkpoint, I would recommend that you keep the Adam state if the dataset is the same (not a transfer learning approach), and thus you should not reset adam's variables.

Btw. if you really want to reset adam, this is how you will do it:

optimizer_reset_op = tf.variables_initializer(optimizer.variables())
sess.run(optimizer_reset_op)