如何安裝時序多頭 model？

Question

我嘗試通過將 2 個模型連接在一起來創建 model。 我想使用的模型應處理時間序列，並且我正在嘗試使用 Conv1D 層。 由於它們有一個 3D 輸入形狀 batch_shape + (steps, input_dim) 和 Keras TimeseriesGenerator 提供這樣的，我很高興能夠在處理單頭模型時使用它。

import pandas as pd
import numpy as np
import random
from sklearn.model_selection import train_test_split
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import (Input, Dense, Conv1D, BatchNormalization, 
                                     Flatten, Dropout, MaxPooling1D,
                                     concatenate)
from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator
from tensorflow.keras.utils import plot_model

data = pd.DataFrame(index=pd.date_range(start='2020-01-01', periods=300, freq='D'))
data['featureA'] = [random.random() for _ in range(len(data))]
data['featureB'] = [random.random() for _ in range(len(data))]
data['featureC'] = [random.random() for _ in range(len(data))]
data['featureD'] = [random.random() for _ in range(len(data))]
data['target']   = [random.random() for _ in range(len(data))]

Xtrain_AB, Xtest_AB, yTrain_AB, yTest_AB = train_test_split(data[['featureA', 'featureB']], 
                                                            data['target'], test_size=0.2,
                                                            shuffle=False)
Xtrain_CD, Xtest_CD, yTrain_CD, yTest_CD = train_test_split(data[['featureC', 'featureD']], 
                                                            data['target'], test_size=0.2,
                                                            shuffle=False)

n_steps = 5 

train_gen_AB = TimeseriesGenerator(Xtrain_AB, yTrain_AB,
                                length=n_steps, 
                                sampling_rate=1,
                                batch_size=64,
                                shuffle=False)
test_gen_AB = TimeseriesGenerator(Xtest_AB, yTest_AB,
                                length=n_steps, 
                                sampling_rate=1,
                                batch_size=64,
                                shuffle=False)

n_features_AB = len(Xtrain_AB.columns)
input_AB      = Input(shape=(n_steps, n_features_AB))
layer_AB      = Conv1D(filters=128, kernel_size=3, activation='relu', input_shape=(n_steps, n_features_AB))(input_AB)
layer_AB      = MaxPooling1D(pool_size=2)(layer_AB)
layer_AB      = Flatten()(layer_AB)
dense_AB      = Dense(50, activation='relu')(layer_AB)
output_AB     = Dense(1)(dense_AB)
model_AB      = Model(inputs=input_AB, outputs=output_AB)
model_AB.compile(optimizer='adam', loss='mse')
model_AB.summary()
model_AB.fit(train_gen_AB, epochs=1, verbose=1)
print(f'evaluation: {model_AB.evaluate(test_gen_AB)}')

#plot_model(model_AB)

train_gen_CD = TimeseriesGenerator(Xtrain_CD, yTrain_CD,
                                length=n_steps, 
                                sampling_rate=1,
                                batch_size=64,
                                shuffle=False)
test_gen_CD = TimeseriesGenerator(Xtest_CD, yTest_CD,
                                length=n_steps, 
                                sampling_rate=1,
                                batch_size=64,
                                shuffle=False)

n_features_CD = len(Xtrain_CD.columns)
input_CD      = Input(shape=(n_steps, n_features_CD))
layer_CD      = Conv1D(filters=128, kernel_size=3, activation='relu', input_shape=(n_steps, n_features_CD))(input_CD)
layer_CD      = MaxPooling1D(pool_size=2)(layer_CD)
layer_CD      = Flatten()(layer_CD)
dense_CD      = Dense(50, activation='relu')(layer_CD)
output_CD     = Dense(1)(dense_CD)
model_CD      = Model(inputs=input_CD, outputs=output_CD)
model_CD.compile(optimizer='adam', loss='mse')
model_CD.summary()
model_CD.fit(train_gen_CD, epochs=1, verbose=1)
print(f'evaluation: {model_CD.evaluate(test_gen_CD)}')

#plot_model(model_CD)

這適用於每個模型:)

現在我想嘗試將兩個模型連接到一個模型（因為我認為它可以讓我稍后添加額外的“頭”來並行訓練它們，我猜使用這樣的模型可能更容易處理很多分離一次）並得到一個雙頭 model，可以像這樣輕松創建

merge=concatenate(inputs=[layer_AB, layer_CD])
dense_merge     = Dense(50, activation='relu')(merge)
output_merge    = Dense(1)(dense_merge)
model_dual_head = Model(inputs=[input_AB, input_CD], outputs=output_merge)
model_dual_head.compile(optimizer='adam', loss='mse')
model_dual_head.summary()
print(f'dual head model input_shape:{model_dual_head.input_shape}')
plot_model(model_dual_head) 

這個 dual_head_model 的 input_shape 為 2 倍 3D [(None, 5, 2), (None, 5, 2)]最終看起來是這樣

不幸的是，我不知道如何適應它:(並希望您能夠為我提供有關如何生成所需數據形狀的解決方案。我嘗試將以前使用的生成器提供為列表model_dual_head.fit([train_gen_AB, train_gen_CD], epochs=1, verbose=1) ，以及原始輸入數據幀model_dual_head.fit(x=[Xtrain_AB, Xtrain_CD], y=[yTrain_AB, yTrain_CD], epochs=1, verbose=1)的列表，但它似乎不是正確的形狀。

提前致謝

瓦西里

---

根據 Jacks 評論，我嘗試使用以下代碼

def doubleGen(gen1, gen2):
  assert(len(gen1) == len(gen2))
  for feature1, label1, feature2, label2 in (train_gen_AB, train_gen_CD):
    yield (feature1, feature2), label1 

gen = doubleGen(train_gen_AB, train_gen_CD)
model_dual_head.fit(gen, epochs=1, verbose=1)

但不幸的是它不起作用，因為 input_shape 不一樣

ValueError: Layer model_2 expects 2 input(s), but it received 4 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, None, None) dtype=float32>, <tf.Tensor 'ExpandDims:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'IteratorGetNext:2' shape=(None, None, None) dtype=float32>, <tf.Tensor 'ExpandDims_1:0' shape=(None, 1) dtype=float32>]

---

我根據 Jacks note 調整了生成器中的括號，現在得到了不同的錯誤。

def doubleGen(gen1, gen2):
  assert(len(gen1) == len(gen2))
  for (feature1, label1), (feature2, label2) in train_gen_AB, train_gen_CD:
    assert label1 == label2
    yield (feature1, feature2), label1 

gen = doubleGen(train_gen_AB, train_gen_CD)
model_dual_head.fit(gen, epochs=1, verbose=1)

<ipython-input-8-e8971cd0f287> in doubleGen(gen1, gen2)
      1 def doubleGen(gen1, gen2):
      2   assert(len(gen1) == len(gen2))
----> 3   for (feature1, label1), (feature2, label2) in train_gen_AB, train_gen_CD:
      4     assert label1 == label2
      5     yield (feature1, feature2), label1

ValueError: too many values to unpack (expected 2)

我考慮過使用一個普通索引迭代生成器來修復形狀主題，但這會導致 NonType 錯誤

def doubleGen(gen1, gen2):
  assert(len(gen1) == len(gen2))
  for i in range(len(gen1)):
    feature1, label1 = gen1[i]
    feature2, label2 = gen2[i]
    #assert label1.all() == label2.all()
    yield (feature1, feature2), label1 

gen = doubleGen(train_gen_AB, train_gen_CD)
model_dual_head.fit(gen, epochs=1, verbose=1)

TypeError                                 Traceback (most recent call last)
<ipython-input-24-6abda48a58c7> in <module>()
      8 
      9 gen = doubleGen(train_gen_AB, train_gen_CD)
---> 10 model_dual_head.fit(gen, epochs=1, verbose=1)

2 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    853       # In this case we have created variables on the first call, so we run the
    854       # defunned version which is guaranteed to never create variables.
--> 855       return self._stateless_fn(*args, **kwds)  # pylint: disable=not-callable
    856     elif self._stateful_fn is not None:
    857       # Release the lock early so that multiple threads can perform the call

TypeError: 'NoneType' object is not callable

---

可以在此處訪問筆記本

---

最后，傑克斯的意見幫助找到了解決方案。 zip 只是缺少兩個生成器，以便能夠迭代它們:)

def doubleGen(gen1, gen2):
  assert(len(gen1) == len(gen2))
  for (feature1, label1), (feature2, label2) in zip(train_gen_AB, train_gen_CD):
    assert label1.all() == label2.all()
    yield (feature1, feature2), label1 

gen = doubleGen(train_gen_AB, train_gen_CD)
model_dual_head.fit(gen, epochs=1, verbose=1)

4/4 [==============================] - 0s 14ms/step - loss: 0.1119
<tensorflow.python.keras.callbacks.History at 0x7f0dfd4de5d0>

Answer 1

我不知道這是否已經存在，但我相信您可以創建一個新的生成器來合並兩個數據集。 假設兩個生成器步調一致，這應該可以工作：

for (input1, label1), (input2, label2) in generator1, generator2:
    assert label1 == label2
    yield (input1, input2), label1

現在，這給出了一個生成器，該生成器將兩個輸入作為一個元組生成，並將公共 label 作為一個數據項。 這可能是 lambda，這樣可以省去創建整個發電機 class 的麻煩。