矩阵尺寸不兼容 - Keras Tensorflow

Question

I'm trying to train a simple model over some picture data that belongs to 10 classes. The images are in B/W format (not gray scale), I'm using the image_dataset_from_directory to import the data into python as well as split it into validation/training sets. My code is as below:

My Imports

import matplotlib.pyplot as plt
import numpy as np
import os
import PIL

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.layers import Dense

Read Image Data

trainDT = tf.keras.preprocessing.image_dataset_from_directory(
    data_path,
    labels="inferred",
    label_mode="categorical",
    class_names=['0','1','2','3','4','5','6','7','8','9'],
    color_mode="grayscale",
    batch_size=4,
    image_size=(256, 256),
    shuffle=True,
    seed=44,
    validation_split=0.1,
    subset='validation',
    interpolation="bilinear",
    follow_links=False,
)

Model Creation/Compile/Fit

model = Sequential([
   Dense(units=128, activation='relu', input_shape=(256,256,1), name='h1'),
   Dense(units=64, activation='relu',name='h2'),
   Dense(units=16, activation='relu',name='h3'),
   layers.Flatten(name='flat'),
   Dense(units=10, activation='softmax',name='out')
],name='1st')

model.summary()

model.compile(optimizer='adam' , loss='categorical_crossentropy', metrics=['accuracy'])

model.fit(x=trainDT, validation_data=train_data, epochs=10, verbose=2)

The model training returns an error:

InvalidArgumentError                      Traceback (most recent call last)
....
    /// anaconda paths and anaconda python code snippets in the error reporting \\\
....
InvalidArgumentError:  Matrix size-incompatible: In[0]: [1310720,3], In[1]: [1,128]
     [[node 1st/h1/Tensordot/MatMul (defined at <ipython-input-38-58d6507e2d35>:1) ]] [Op:__inference_test_function_11541]

Function call stack:
test_function

I don't understand where the size mismatch comes from, I've spent a few hours looking around for a solution and trying different things but nothing seems to work for me. Appreciate any help, thank you in advance!

Answer 1

Hy mhk777 Hope you are doing well. Brother, I think that you are confusing dense layers with convolution layers. You have to apply some convolution layers to the image before giving it to dense layers. If you don't want to apply convolution than you have to give 2d array to the dense layer ie (number of samples, data)

import tensorflow as tf
from tensorflow.keras import datasets, layers, models

model = models.Sequential()
# Here are convolutional Layer
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(256,256,1)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))

# Here are your dense layers
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10))

model.summary()

model.compile(optimizer='adam' , loss='categorical_crossentropy', metrics=['accuracy'])

model.fit(x=trainDT, validation_data=train_data, epochs=10, verbose=2)

Answer 2

Dense layers expect flat input (not 3d tensor), but you are sending (256,256,1) shaped tensor into the first dense layer. If you want to use dense layers from the beginning then you will need to move the flatten to be the first layer or you will need to properly reshape your data.

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dense(10, activation="softmax")
])

Also, the flatten between 2 dense layers makes no sense because the output of a dense layer is flat anyway.

From the structure of your model (especially the flatten placement), I assume that those dense layers were supposed to be convolutional layers instead.

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, (3, 3), activation='relu'),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dense(10, activation="softmax")
])

Convolutional layers can process 2D input and they will also produce more dimensional output which you need to flatten before passing it to the dense top (note that you can add more convolutional layers).