TensorFlow示例，但具有中间层

Question

I am trying to get this code to work. It may not look like it, but it comes mostly from the TensorFlow mnist example. I am trying to get three layers, though, and I've changed the input and output size. The input size is 12, the mid size is 6, and the output size is 2. This is what happens when I run this. It does not throw an error, but when I run the test option I always get 50%. When I go back to training it runs and I am sure the weights are changing. There is code for saving the model and weights, so I'm pretty confident it's not wiping out my weights every time I re-start it. The idea behind self.d_y_out is to have something that will allow me to run the model and get output for just one image. I think the problem is near the comment that says "PROBLEM??".

        self.d_keep = tf.placeholder(tf.float32)
        self.d_W_2 = tf.Variable(tf.random_normal([mid_num, output_num], stddev=0.0001))
        self.d_b_2 = tf.Variable(tf.random_normal([output_num], stddev=0.5))

        self.d_x = tf.placeholder(tf.float32, [None, input_num])
        self.d_W_1 = tf.Variable(tf.random_normal([input_num, mid_num], stddev=0.0001))  # 0.0004
        self.d_b_1 = tf.Variable(tf.zeros([mid_num]))

        self.d_y_ = tf.placeholder(tf.float32, [None, output_num])

        self.d_x_drop = tf.nn.dropout(self.d_x, self.d_keep)

        self.d_y_logits_1 = tf.matmul(self.d_x_drop, self.d_W_1) + self.d_b_1
        self.d_y_mid = tf.nn.relu(self.d_y_logits_1) 
        self.d_y_mid_drop = tf.nn.dropout(self.d_y_mid, self.d_keep)

        self.d_y_logits_2 = tf.matmul(self.d_y_mid_drop, self.d_W_2) + self.d_b_2

        self.d_y_softmax = tf.nn.softmax_cross_entropy_with_logits(logits=self.d_y_logits_2, labels=self.d_y_)

        self.d_cross_entropy = tf.reduce_mean(self.d_y_softmax) ## PROBLEM??

        self.d_train_step = tf.train.GradientDescentOptimizer(0.001).minimize(self.d_cross_entropy)  # 0.0001

        # train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) #0.5

        #self.d_y_out = tf.argmax(self.d_y, 1)  ## for prediction
        self.d_y_out = tf.argmax(self.d_y_logits_2, 1, name="d_y_out")

    if self.train :

        for i in range(self.start_train,self.cursor_tot): #1000
            batch_xs, batch_ys = self.get_nn_next_train(self.batchsize)
            self.sess.run(self.d_train_step, feed_dict={self.d_x: batch_xs, self.d_y_: batch_ys, self.d_keep: 0.5})
            if True: #mid_num > 0:
                cost = self.sess.run([self.d_cross_entropy, self.d_train_step], 
                    feed_dict={self.d_x: batch_xs, self.d_y_: batch_ys, self.d_keep: 0.5})
                print cost[0], "cost"


    if self.test :
        d_correct_prediction = tf.equal(self.d_y_out, tf.argmax(self.d_y_,1))
        #d_correct_prediction = tf.equal(tf.argmax(self.d_y , 1), tf.argmax(self.d_y_, 1))

        d_accuracy = tf.reduce_mean(tf.cast(d_correct_prediction, tf.float32))

        if self.use_loader : self.get_nn_next_test(self.batchsize)
        print(self.sess.run([d_accuracy, self.d_cross_entropy], 
            feed_dict={self.d_x: self.mnist_test.images, self.d_y_: self.mnist_test.labels, self.d_keep: 1.0}))

    if self.predict_dot :
        for i in range(start, stop ) :
            batch_0, batch_1 = self.get_nn_next_predict(self.batchsize)
            if len(batch_0) > 0 :
                out.extend( self.sess.run([self.d_y_out, self.d_cross_entropy], 
                    feed_dict={self.d_x : batch_0, self.d_y_: batch_1, self.d_keep: 1.0})[0])
                print "out" , len(out) , i, self.cursor_tot, out[:10],"..."

EDIT I've edited the code in this question significantly. Much thanks to vijay m for getting me this far. Any help would be appreciated. Thanks.

Answer 1

The problem in this code is you calling dropout on the inputs. Yours is a single layer network and you don't need dropout . And use a momentum optimizer like Adam for training faster. The changes i made:

d_y_logits_1 = tf.matmul(d_x, d_W_1) + d_b_1
d_y_mid = tf.nn.relu(d_y_logits_1) 

d_y_logits_2 = tf.matmul(d_y_mid, d_W_2) + d_b_2

d_y_softmax = tf.nn.softmax_cross_entropy_with_logits(logits=d_y_logits_2, labels=d_y_)

d_cross_entropy = tf.reduce_mean(d_y_softmax)

d_train_step = tf.train.AdamOptimizer(0.01).minimize(d_cross_entropy)