自动编码器在训练时不学习

Question

python 3.5.2, tensorflow 1.0.0

Somewhat new in programming with autoencoders. I am trying to implement a simple network to get familiarize from here . I have used the same input data in which a CNN is able to classify perfectly with accuracy of 98%. My data have 2000 row data and each row is a signal. I am trying with 3 stacked layers of auto encoders with 512 256 and 64 nodes.

class dimensions:
input_width, input_height = 1,1024
BATCH_SIZE = 50
layer = [input_width*input_height, 512, 256, 64]
learningrate = 0.001

def myencoder(x,corrupt_prob,dimensions):
current_input = corrupt(x) * corrupt_prob + x * (1 - corrupt_prob)
encoder = []
for layer_i, n_output in enumerate(dimensions.layer[1:]):
    n_input = int(current_input.get_shape()[1])
    W = tf.Variable(
        tf.random_uniform([n_input, n_output],
                          -1.0 / math.sqrt(n_input),
                          1.0 / math.sqrt(n_input)))
    b = tf.Variable(tf.zeros([n_output]))
    encoder.append(W)
    output = tf.nn.tanh(tf.matmul(current_input, W) + b)

    current_input = output

z = current_input
encoder.reverse()
# Build the decoder using the same weights
for layer_i, n_output in enumerate(model.layer[:-1][::-1]):
    W = tf.transpose(encoder[layer_i])
    b = tf.Variable(tf.zeros([n_output]))
    output = tf.nn.tanh(tf.matmul(current_input, W) + b)

    current_input = output
# now have the reconstruction through the network
y = current_input
# cost function measures pixel-wise difference
cost = tf.sqrt(tf.reduce_mean(tf.square(y - x)))

return z,y,cost

sess = tf.Session()
model = dimensions()
data_train,data_test,label_train,label_test = load_data(Datainfo,folder)

x = tf.placeholder(tf.float32,[model.BATCH_SIZE,model.input_height*model.input_width])
corrupt_prob = tf.placeholder(tf.float32,[1])
z,y,cost = myencoder(x,corrupt_prob,dimensions)
train_step = tf.train.AdamOptimizer(model.learningrate).minimize(cost)
lossfun = np.zeros(STEPS)
sess.run(tf.global_variables_initializer())

for i in range(STEPS):
    train_data = batchdata(data_train, model.BATCH_SIZE)
    epoch_loss = 0
    for j in range(model.BATCH_SIZE):
        sess.run(train_step,feed_dict={x:train_data,corrupt_prob:[1.0]})
        c = sess.run(cost, feed_dict={x: train_data, corrupt_prob: [1.0]})
        epoch_loss += c
    lossfun[i] = epoch_loss
    print('Epoch', i, 'completed out of', STEPS, 'loss:', epoch_loss)

my loss function appears like this xaxis - no of iterations, y axis - loss

the loss doesn't decrease and the network doesn't learn anything. any help appreciated !

Answer 1

在函数myencoder中，权重变量W和b在每个训练步骤中都会初始化。