ValueError: Input 0 of layer "sequential" is incompatible with the layer: expected shape=(None, 128, 128, 3), found shape=(32, 128, 3)

Question

i have the following code portion where i used the vit_b16 model. The input to the model is a 128x128x3 Multi-spectral image.

!pip install vit-keras
!pip install tensorflow_addons
from vit_keras import vit, utils
IMG_SIZE = (128,128)
vit_base_model =  vit.vit_b16(image_size=IMG_SIZE,pretrained=True,include_top=False,pretrained_top=False)
vit_model = Model(inputs=vit_base_model.input, outputs=vit_base_model.layers[18].output)
model=keras.models.Sequential()
model.add(vit_model)
model.add(Flatten())
model.add(Dense(226))
model.add(Dropout(0.5))
model.add(Dense(226))
model.summary()
model.compile(
     optimizer=keras.optimizers.Adam(),
     loss=keras.losses.BinaryCrossentropy(from_logits=True), 
     metrics=[keras.metrics.BinaryAccuracy()],
)
epochs = 20
model.fit(Ref_L7,hyp_patches,epochs=epochs, validation_data=0.1)

I am getting this error from the model.compile part.

Answer 1

The problem is with your size of data, and you can try with this, also your shape of data in fit should be (numberofImages,128,128,3) .

IMG_SIZE = (128,128,3)
vit_base_model =  vit.vit_b16(image_size=IMG_SIZE,pretrained=True,include_top=False,pretrained_top=False)
vit_model = Model(inputs=vit_base_model.input, outputs=vit_base_model.layers[18].output)
model=keras.models.Sequential()
model.add(vit_model)
model.add(Flatten())
model.add(Dense(226))
model.add(Dropout(0.5))
model.add(Dense(226))
model.summary()
model.compile(
     optimizer=keras.optimizers.Adam(),
     loss=keras.losses.BinaryCrossentropy(from_logits=True), 
     metrics=[keras.metrics.BinaryAccuracy()],
)
epochs = 20
model.fit(Ref_L7,hyp_patches,epochs=epochs, validation_data=0.1)

Changed the IMAGE_SIZE as channel 3 is added. also print your shape of Ref_L7,hyp_patches This will give you more information on what's wrong.