sklearn MLPRegressor的Tensorflow副本產生其他結果

Question

我試圖在Tensorflow中重現深度學習回歸結果。 如果我使用sklearn中的MLPRegressor類訓練神經網絡，我會得到98％驗證的非常好的結果。

MLPRegressor：

http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html#sklearn.neural_network.MLPRegressor

我試圖在Tensorflow中重現該模型。 通過在Tensorflow模型中復制MLPRegressor類的默認值。 但是我無法得到相同的結果。 我大部分時間只得到75％。

我的TF型號：

tf.reset_default_graph()
graph = tf.Graph()
n_input = 3  # n variables
n_hidden_1 = 100
n_hidden_2 = 1
n_output = 1
beta = 0.001

learning_rate = 0.001

with graph.as_default():
    tf_train_feat = tf.placeholder(tf.float32, shape=(None, n_input))
    tf_train_label = tf.placeholder(tf.float32, shape=(None))


    tf_test_feat = tf.constant(test_feat, tf.float32)


    """
    Weights and biases. The weights matix' columns will be the output vector.

    * ndarray([rows, columns])
    * ndarray([in, out])

    tf.placeholder(None) and tf.placeholder([None, 3]) means that the row's size is not set. In the second 
    placeholder the columns are prefixed at 3.
    """
    W = {
        "layer_1": tf.Variable(tf.truncated_normal([n_input, n_hidden_1])),
        "layer_2": tf.Variable(tf.truncated_normal([n_hidden_1, n_hidden_2])),
        "layer_3": tf.Variable(tf.truncated_normal([n_hidden_2, n_output])),
    }


    b = {
        "layer_1": tf.Variable(tf.zeros([n_hidden_1])),
        "layer_2": tf.Variable(tf.zeros([n_hidden_2])),
    }

    def computation(X):
        layer_1 = tf.nn.relu(tf.matmul(X, W["layer_1"]) + b["layer_1"])
        layer_2 = tf.nn.relu(tf.matmul(layer_1, W["layer_2"]) + b["layer_2"])
        return layer_2

    tf_prediction = computation(tf_train_feat)
    tf_test_prediction = computation(tf_test_feat)

    tf_loss = tf.reduce_mean(tf.pow(tf_train_label - tf_prediction, 2)) 
    tf_loss = tf.reduce_mean( tf_loss + beta * tf.nn.l2_loss(W["layer_2"]) )
    tf_optimizer = tf.train.AdamOptimizer(learning_rate).minimize(tf_loss)
    #tf_optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(tf_loss)

    init = tf.global_variables_initializer()

我的TF會議：

def accuracy(y_pred, y):
    a = 0
    for i in range(y.shape[0]):
        a += abs(1 - y_pred[i][0] / y[i])

    return round((1 - a / y.shape[0]) * 100, 3)

def accuracy_tensor(y_pred, y):
    a = 0
    for i in range(y.shape[0]):
        a += abs(1 - y_pred[i][0] / y[i])

    return round((1 - a / y.shape[0]) * 100, 3)

# Shuffles two arrays.
def shuffle_in_unison(a, b):
    assert len(a) == len(b)
    shuffled_a = np.empty(a.shape, dtype=a.dtype)
    shuffled_b = np.empty(b.shape, dtype=b.dtype)
    permutation = np.random.permutation(len(a))
    for old_index, new_index in enumerate(permutation):
        shuffled_a[new_index] = a[old_index]
        shuffled_b[new_index] = b[old_index]
    return shuffled_a, shuffled_b

train_epoch = int(5e4)
batch = int(200)

n_batch = int(X.shape[0] // batch)

prev_acc = 0
stable_count = 0

session = tf.InteractiveSession(graph=graph)
session.run(init)
print("Initialized.\n No. of epochs: %d.\n No. of batches: %d." % (train_epoch, n_batch))

for epoch in range(train_epoch):
    offset = (epoch * n_batch) % (Y.shape[0] - n_batch)


    for i in range(n_batch):
        x = X[offset:(offset + n_batch)]
        y = Y[offset:(offset + n_batch)]

        x, y = shuffle_in_unison(x, y)

        feed_dict = {tf_train_feat: x, tf_train_label: y}
        _, l, pred, pred_label = session.run([tf_optimizer, tf_loss, tf_prediction, tf_train_label], feed_dict=feed_dict)

    if epoch % 1 == 0:
        print("Epoch: %d. Batch' loss: %f" %(epoch, l))
        test_pred = tf_test_prediction.eval(session=session)

        acc_test = accuracy(test_pred, test_label)
        acc_train = accuracy_tensor(pred, pred_label)

        print("Accuracy train set %s%%" % acc_train)
        print("Accuracy test set: %s%%" % acc_test)

我在Tensorflow代碼中遺漏了什么嗎？ 謝謝！

Answer 1

除非你有充分的理由不使用它們，否則回歸應該有線性輸出單位。 我一段時間遇到了類似的問題，結果使用了線性輸出和線性隱藏單元，這似乎反映了我的情況下的mlpregressor。

在第6章中，Goodfellow的深度學習書中有一個很棒的部分，從第181頁開始，它涉及激活功能。

至少嘗試一下輸出層

 layer_2 = tf.matmul(layer_1, W["layer_2"]) + b["layer_2"]