How to use Keras Conv2D layer with variably shaped input

Question

I've got a NP array called X_train with the following properties:

X_train.shape = (139,)
X_train[0].shape = (210, 224, 3)
X_train[1].shape = (220,180, 3)

In other words, there are 139 observations. Each image has a different width and height, but they all have 3 channels. So the dimension should be (139, None, None, 3) where None = variable.

Since you don't include the dimension for the number of observations in the layer, for the Conv2D layer I used input_shape=(None,None,3) . But that gives me the error:

expected conv2d_1_input to have 4 dimensions, but got array with shape (139, 1)

My guess is that the problem is that the input shape is (139,) instead of (139, None, None, 3) . I'm not sure how to convert to that however.

Answer 1

One possible solution to your problem is to fill the arrays with zeros so that they all have a similar size. Afterwards, your input shape will be something like (139, max_x_dimension, max_y_dimension, 3) .

The following functions will do the job:

import numpy as np

def fillwithzeros(inputarray, outputshape):
    """
    Fills input array with dtype 'object' so that all arrays have the same shape as 'outputshape'
    inputarray: input numpy array
    outputshape: max dimensions in inputarray (obtained with the function 'findmaxshape')

    output: inputarray filled with zeros
    """
    length = len(inputarray)
    output = np.zeros((length,)+outputshape, dtype=np.uint8)
    for i in range(length):
        output[i][:inputarray[i].shape[0],:inputarray[i].shape[1],:] = inputarray[i]
    return output

def findmaxshape(inputarray):
    """
    Finds maximum x and y in an inputarray with dtype 'object' and 3 dimensions
    inputarray: input numpy array

    output: detected maximum shape
    """
    max_x, max_y, max_z = 0, 0, 0
    for array in inputarray:
        x, y, z = array.shape
        if x > max_x:
            max_x = x
        if y > max_y:
            max_y = y
        if z > max_z:
            max_z = z
    return(max_x, max_y, max_z)

#Create random data similar to your data
random_data1 = np.random.randint(0,255, 210*224*3).reshape((210, 224, 3))
random_data2 = np.random.randint(0,255, 220*180*3).reshape((220, 180, 3))
X_train = np.array([random_data1, random_data2])

#Convert X_train so that all images have the same shape
new_shape = findmaxshape(X_train)
new_X_train = fillwithzeros(X_train, new_shape)