如何使用Keras在Dense层中使用Dropout创建自动编码器

Question

The Concept:

I am trying to reconstruct the output of a numeric dataset, for which I'm trying different approaches on Autoencoders. One of the approach is to use dropout in Dense layers.

The Problem:

As shown the 2 parts where encoder and decoder are used, the dimension diminishes in the center. That's where the problem begins since the Dense Layers with Dropout do not pickup.

Note that this is my Second way of trying Autoencoders, I've already accomplished as shown here .

Here's how I've (Naively) written:

from keras import models
from keras import layers
from keras import backend as K

network = models.Sequential()
input_shape = x_train_clean.shape[1]   # input_shape = 3714

outer_layer = int(input_shape / 7)
inner_layer = int(input_shape / 14)

network.add(Dropout(0.2, input_shape=(input_shape,)))

network.add(Dense(units=outer_layer, activation='relu'))

network.add(Dropout(0.2))

network.add(Dense(units=inner_layer, activation='relu'))

network.add(Dropout(0.2))

network.add(Dense(units=10, activation='linear'))

network.add(Dropout(0.2))

network.add(Dense(units=inner_layer, activation='relu'))

network.add(Dropout(0.2))

network.add(Dense(units=outer_layer, activation='relu'))

network.add(Dropout(0.2))

network.compile(loss=lambda true, pred: K.sqrt(K.mean(K.square(pred-true))),  # RMSE
                  optimizer='rmsprop',  # Root Mean Square Propagation
                  metrics=['accuracy'])  # Accuracy performance metric

history = network.fit(x_train_noisy,  # Features
                        x_train_clean,  # Target vector
                        epochs=3,  # Number of epochs
                        verbose=0,  # No output
                        batch_size=100,  # Number of observations per batch
                        shuffle=True,)   # training data will be randomly shuffled at each epoch

The Output:

Output Error states it very clear:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [100,530] vs. [100,3714] [[{{node loss_1/dropout_9_loss/sub}}]] It is unable to pickup from lower dimension to higher dimension.

Open Questions:

1. Is it even possible to use Dropout for Autoencoder
2. What possible layers can I try, if Sequential is the problem

Answer 1

The error you're seeing has nothing to do with the learning capacity of the network. network.summary() reveals that the output shape is (None, 530) while the input shape is (None, 3714) leading to the error while training.

Inputs that cause error during training:

x_train_noisy = np.zeros([100, 3714]) #just to test
x_train_clean = np.ones([100, 3714])
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [100,530] vs. [100,3714]

Inputs that train without error:

x_train_noisy = np.zeros([100, 3714]) #just to test
x_train_clean = np.ones([100, 530])

100/100 [==============================] - 1s 11ms/step - loss: 1.0000 - acc: 1.0000