计算验证损失与训练损失同时进行

Question

I am trying to compute the validation loss on each iteration after the weights are computed (on the train set). How can I use the resulted weight tensor to predict the values on the validation set?

I tried using two arrays to store the loss values at each step of the session.

X_tr, X_val, y_tr, y_val = train_test_split(train_set, y_train, test_size=0.2, random_state=42)

x = tf.placeholder(tf.float32, X_tr.shape, name = 'data')
y = tf.placeholder(tf.float32, y_tr.shape, name = 'labels')
W = tf.Variable(tf.zeros([len(train_set.columns),1]), dtype = tf.float32, name = 'weights')
b = tf.Variable(0, dtype = tf.float32, name = 'bias')
y_pred = tf.matmul(x, W) + b

loss = tf.reduce_mean(tf.square(y - y_pred), name = 'loss')
optimizer = tf.train.MomentumOptimizer(learning_rate = 0.0006, momentum = 0.90)
train_op = optimizer.minimize(loss)

losses_t, losses_v = [], []
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    for i in range(1000):
        _, loss_t = sess.run([train_op, loss], feed_dict = {x: X_tr, y: y_tr})
        losses_t.append(loss_t)
        loss_v = sess.run(loss, feed_dict = {x: X_val, y: y_val})
        losses_v.append(loss_v)
        if(i % 20 == 0):
            print('Training loss is: ', loss_t)
            print('Validation loss is: ', loss_v)

    W_value, b_value = sess.run([W, b])

The error:

ValueError: Cannot feed value of shape (292, 220) for Tensor 'data_6:0', which has shape '(1166, 220)'

Answer 1

The problem was that I was giving the same parameters to both sess.run functions. I created instead new placeholders (to fit the shape of the input X_val and y_val) and a loss function for the validation loss specific to its test set (X_val, y_val).

losses_t = []
losses_v = []
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    for i in range(3000):
        _, loss_t = sess.run([train_op, loss], feed_dict = {x: X_tr, y: y_tr})
        losses_t.append(loss_t)
        loss_v = sess.run(lossv, feed_dict = {xv: X_val, yv: y_val})
        losses_v.append(loss_v)
        if(i % 20 == 0):
            print('Training loss is: ', loss_t)
            print('Validation loss is: ', loss_v)

    W_value, b_value = sess.run([W, b])