What should I change input_shape to?

Question

I'm going to learn data with 3Dtensor input.

My model is

from keras.models import Sequential
from keras.layers import Dense, InputLayer
import tensorflow as tf
from tensorflow import keras

class AnomalyDetector(Model):
    def __init__(self):
        super(AnomalyDetector, self).__init__()
        self.encoder = tf.keras.Sequential([
            tf.keras.layers.Dense(32,input_shape=(36,501), activation= "relu"),
            tf.keras.layers.Dense(16, activation= "relu"),
            tf.keras.layers.Dense(8, activation= "relu")
        ])
        self.decoder = tf.keras.Sequential([
            tf.keras.layers.Dense(16,input_shape=(36,501), activation= "relu"),
            tf.keras.layers.Dense(32, activation= "relu"),
            tf.keras.layers.Dense(140, activation= "sigmoid")
        ])
    
    def call(self,x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded
model = AnomalyDetector()

The shape of the 3dtensor is

(1500, 36, 501)

model.compile(optimizer='adam', loss='mae', metrics=['accuracy'])
model.fit(train_X, train_y, epochs=100, batch_size = 512, validation_data=(vali_X, vali_y), shuffle=True)

and Error is...

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [155], in <cell line: 12>()
      9         print(e)
     11 model.compile(optimizer='adam', loss='mae', metrics=['accuracy'])
---> 12 model.fit(train_X, train_y, epochs=100, batch_size = 512, validation_data=(vali_X, vali_y), shuffle=True)

.
.
.
   

     ValueError: Input 0 of layer "sequential_58" is incompatible with the layer: expected shape=(None, 36, 501), found shape=(None, 36, 8)
    
    
    Call arguments received by layer "anomaly_detector_30" (type AnomalyDetector):
      • x=tf.Tensor(shape=(None, 36, 501), dtype=float32)

I already change encoder's Dense 8 to 501. but another error occurred.

How should I change this?

Answer 1

1. Dense doesn't take 3d input it takes 1d input for example: input_shape = (,40) here 40 is the number of columns and i left a space because at start your model dosen't knows the length of data that's why in model.summary() input_shape = (None,40) comes.

2. Is your input an image?? if yes than Dense dosen't work on image use Conv2D or Conv3D as per your input data.

If i am wrong and Dense layer takes 3d input, correct me and please provide the link where you read about it.Thanks

Answer 2

Your problem is in the input of your decoder. The decoder sets to get tensors of size (36,501) but it gets the output of the encoder that is a tensor of size (36,8) if you change your decoder input your code work:

class AnomalyDetector(tf.keras.Model):
    def __init__(self):
        super(AnomalyDetector, self).__init__()
        self.encoder = tf.keras.Sequential([
            tf.keras.layers.Dense(32,input_shape=(36,501), activation= "relu"),
            tf.keras.layers.Dense(16, activation= "relu"),
            tf.keras.layers.Dense(8, activation= "relu")
        ])
        self.decoder = tf.keras.Sequential([
            tf.keras.layers.Dense(16,input_shape=(36,8), activation= "relu"),
            tf.keras.layers.Dense(32, activation= "relu"),
            tf.keras.layers.Dense(140, activation= "sigmoid")
        ])
    
    def call(self,x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded

But be careful your encoder just embeds the second size of (Y) your image. I think you should flatten the image for your first layer which means something like this:

class AnomalyDetector(tf.keras.Model):
    def __init__(self):
        super(AnomalyDetector, self).__init__()
        self.encoder = tf.keras.Sequential([
            tf.keras.layers.Flatten(),
            tf.keras.layers.Dense(32, activation= "relu"),
            tf.keras.layers.Dense(16, activation= "relu"),
            tf.keras.layers.Dense(8, activation= "relu")
        ])
        self.decoder = tf.keras.Sequential([
            tf.keras.layers.Dense(16, activation= "relu"),
            tf.keras.layers.Dense(32, activation= "relu"),
            tf.keras.layers.Dense(140, activation= "sigmoid")
        ])
    
    def call(self,x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded