TensorFlow Googlenet 啟動結果不佳

Question

我正在嘗試實現一個版本的 Googlenet 初始神經網絡，但是我使用MNIST 數據集獲得了 10% 的准確度。 這令人擔憂，因為對於簡單的神經網絡，我應該為這個數據集獲得 97+% 的准確率。 所以我確信我沒有正確實現初始神經網絡。 我在下面包含了我的代碼。

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
import tensorflow as tf

x  = tf.placeholder(dtype = tf.float32, shape = [None,784])
y_ = tf.placeholder(dtype = tf.float32, shape = [None,10])

x_input = tf.reshape(x,[-1,28,28,1])


# 1x1 Convolution
W1x1 = tf.Variable(tf.random_normal([1,1,1,1]))
b1x1 = tf.Variable(tf.random_normal([1]))
output1x1 = tf.add(tf.nn.conv2d(x_input,W1x1, strides = [1,1,1,1], padding = 'SAME'),b1x1)
output1x1 = tf.nn.relu(output1x1)


# 5x5 Convolution
W5x5 = tf.Variable(tf.random_normal([1,1,1,1]))
b5x5 = tf.Variable(tf.random_normal([1]))
output5x5 = tf.add(tf.nn.conv2d(output1x1,W5x5, strides = [1,1,1,1], padding = 'SAME'),b5x5)
output5x5 = tf.nn.relu(output5x5)


# 3x3 Convolution
W3x3 = tf.Variable(tf.random_normal([1,1,1,1]))
b3x3 = tf.Variable(tf.random_normal([1]))
output3x3 = tf.add(tf.nn.conv2d(output1x1,W3x3, strides = [1,1,1,1], padding = 'SAME'),b3x3)
output3x3 = tf.nn.relu(output3x3)


# AveragePooling followed by 1x1 convolution
outputPool = tf.nn.avg_pool(output1x1, ksize = [1,2,2,1], strides = [1,1,1,1], padding = "SAME")
Wo1x1 = tf.Variable(tf.random_normal([1,1,1,1]))
bo1x1 = tf.Variable(tf.random_normal([1]))
outputo1x1 = tf.add(tf.nn.conv2d(outputPool,Wo1x1, strides = [1,1,1,1], padding = 'SAME'),bo1x1)
outputo1x1 = tf.nn.relu(outputo1x1)


# Concatonate the 4 convolution products
finalouput = tf.concat([output1x1, output5x5, output3x3, outputo1x1], 3)
finalouput = tf.reshape(finalouput, [-1, 7*7*64])

#Add a fully connected layer
W_fc = tf.Variable(tf.random_normal([7*7*64,1024]))
b_fc = tf.Variable(tf.random_normal([1024]))  
output_fc = tf.add(tf.matmul(finalouput,W_fc), b_fc )
output_fc = tf.nn.relu(output_fc)
output_fc = tf.nn.dropout(output_fc, keep_prob = 0.85)

#Final layer
W_final = tf.Variable(tf.random_normal([1024,10]))
b_final = tf.Variable(tf.random_normal([10]))
predictions = tf.add(tf.matmul(output_fc,W_final), b_final)


# Train the model
cost = tf.reduce_sum(tf.nn.softmax_cross_entropy_with_logits(labels = y_  ,logits = predictions))
optimiser = tf.train.AdamOptimizer(1e-3).minimize(cost)
correct_prediction = tf.equal(tf.argmax(predictions, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(20000):
        batch = mnist.train.next_batch(50)
        if i % 100 == 0:
            train_accuracy = accuracy.eval(feed_dict={x: batch[0], y_: batch[1]})
            print('step %d, training accuracy %g' % (i, train_accuracy))
        optimiser.run(feed_dict={x: batch[0], y_: batch[1]})
    print('test accuracy %g' % accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels,}))

Answer 1

問題在於權重初始化。 使用tf.random_normal()初始化的權重的標准偏差為1 ，這是很高的，減少數量應該可以解決問題。

將權重初始化更改為：

W** = tf.Variable(tf.random_normal(..., stddev=0.01))
b** = tf.Variable(tf.random_normal(..., stddev=0.001))

Answer 2

你的模型很影子。 GoogLeNet 有 22 層。

我不建議自己實現層，因為它容易出錯。 最好使用 tensorflow 層抽象。 您可能還想查看或使用現有的實現，例如， 這里。

Answer 3

也許嘗試以不同的順序連接它？