使用 CNN+LSTM 模型和 TimeDistributed 層包裝器進行 Keras 時間序列預測

Question

我有幾個人類活動識別數據的數據文件，由按時間順序排列的記錄原始樣本行組成。 每行有 8 列 EMG 傳感器數據和 1 個對應的目標傳感器數據列。 我正在嘗試將 8 個通道的 EMG 傳感器數據輸入到 CNN+LSTM 深度模型中，以便預測 1 個通道的目標數據。 為此，我將數據集（下圖中的a ）分解為 50 行原始樣本窗口（下圖中的b ），然后將這些窗口重塑為 4 個窗口的塊，作為 LSTM 部分的時間步長模型（下圖中的c ）。 下圖有望更好地解釋它：

我一直在關注如何實現我的模型的教程： https : //medium.com/smileinnovation/how-to-work-with-time-distributed-data-in-a-neural-network-b8b39aa4ce00

我已經重塑了數據並構建了模型，但不斷返回以下錯誤，我無法弄清楚如何解決：

    "ValueError: Error when checking target: expected FC_out to have 2 dimensions, but got array with shape (808, 50, 1)"

我的代碼如下，是使用 Keras 和 Tensorflow 用 Python 編寫的：

    from keras.models import Sequential
    from keras.layers import CuDNNLSTM
    from keras.layers.convolutional import Conv2D
    from keras.layers.core import Dense, Dropout
    from keras.layers import Flatten
    from keras.layers import TimeDistributed

    #Code that reads in file data and shapes it into 4-window blocks omitted. That code produces the following arrays:
    #x_train  - shape of (808, 4, 50, 8) which equates to (samples, time steps, window length, number of channels)
    #x_valid  - shape of (223, 4, 50, 8) which equates to the same as x_train
    #y_train  - shape of (808, 50, 1) which equates to (samples, window length, number of target channels)


    # Followed machine learning mastery style for ease of reading
    numSteps = x_train.shape[1]
    windowLength = x_train.shape[2]
    numChannels = x_train.shape[3]
    numOutputs = 1

    # Reshape x data for use with TimeDistributed wrapper, adding extra dimension at the end
    x_train = x_train.reshape(x_train.shape[0], numSteps, windowLength, numChannels, 1)
    x_valid = x_valid.reshape(x_valid.shape[0], numSteps, windowLength, numChannels, 1)


    # Build model
    model = Sequential()
    model.add(TimeDistributed(Conv2D(64, (3,3), activation=activation, name="Conv2D_1"), 
                                     input_shape=(numSteps, windowLength, numChannels, 1)))

    model.add(TimeDistributed(Conv2D(64, (3,3), activation=activation, name="Conv2D_2")))
    model.add(Dropout(0.4, name="CNN_Drop_01"))

    # Flatten for passing to LSTM layer
    model.add(TimeDistributed(Flatten(name="Flatten_1")))

    # LSTM and Dropout
    model.add(CuDNNLSTM(28, return_sequences=True, name="LSTM_01"))
    model.add(Dropout(0.4, name="Drop_01"))

    # Second LSTM and Dropout
    model.add(CuDNNLSTM(28, return_sequences=False, name="LSTM_02"))
    model.add(Dropout(0.3, name="Drop_02"))

    # Fully Connected layer and further Dropout
    model.add(Dense(16, activation=activation, name="FC_1"))
    model.add(Dropout(0.4)) # For example, for 3 outputs classes 

    # Final fully Connected layer specifying outputs
    model.add(Dense(numOutputs, activation=activation, name="FC_out"))

    # Compile model, produce summary and save model image to file
    # NOTE: coeffDetermination refers to a function for calculating R2 and is not included in this code
    model.compile(optimizer='Adam', loss='mse', metrics=[coeffDetermination])


    # Now train the model
    history_cb = model.fit(x_train, y_train, validation_data=(x_valid, y_valid), epochs=30, batch_size=64)

如果有人能弄清楚我做錯了什么，我將不勝感激。 或者我只是以不正確的方式解決這個問題，嘗試使用此模型配置進行時間序列預測？

Answer 1

“ValueError：檢查目標時出錯：預期 FC_out 有 2 維，但得到形狀為 (808, 50, 1) 的數組”

• 你的輸入是 (808, 4, 50, 8, 1) 輸出是 (808, 50, 1)
• 但是，從 model.summary() 顯示輸出形狀應該是 (None, 4, 1)
• 由於時間步數為 4，因此 y_train 應該類似於 (808, 4, 1)。
• 或者，如果您想要 (888, 50, 1)，則需要更改模型以將最后一部分設為 (None, 50, 1)。

Model: "sequential_10"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
time_distributed_18 (TimeDis (None, 4, 48, 6, 64)      640       
_________________________________________________________________
time_distributed_19 (TimeDis (None, 4, 46, 4, 64)      36928     
_________________________________________________________________
CNN_Drop_01 (Dropout)        (None, 4, 46, 4, 64)      0         
_________________________________________________________________
time_distributed_20 (TimeDis (None, 4, 11776)          0         
_________________________________________________________________
LSTM_01 (LSTM)               (None, 4, 28)             1322160   
_________________________________________________________________
Drop_01 (Dropout)            (None, 4, 28)             0         
_________________________________________________________________
Drop_02 (Dropout)            (None, 4, 28)             0         
_________________________________________________________________
FC_1 (Dense)                 (None, 4, 16)             464       
_________________________________________________________________
dropout_3 (Dropout)          (None, 4, 16)             0         
_________________________________________________________________
FC_out (Dense)               (None, 4, 1)              17        
=================================================================
Total params: 1,360,209
Trainable params: 1,360,209
Non-trainable params: 0

對於具有不同序列長度的多對多序列預測，請查看此鏈接https://github.com/keras-team/keras/issues/6063

dataX or input : (nb_samples, nb_timesteps, nb_features) -> (1000, 50, 1)
dataY or output: (nb_samples, nb_timesteps, nb_features) -> (1000, 10, 1)

model = Sequential()  
model.add(LSTM(input_dim=1, output_dim=hidden_neurons, return_sequences=False))  
model.add(RepeatVector(10))
model.add(LSTM(output_dim=hidden_neurons, return_sequences=True))  
model.add(TimeDistributed(Dense(1)))
model.add(Activation('linear'))   
model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['accuracy'])