ValueError: Error when checking target: expected avg_pool to have 4 dimensions, but got array with shape (100, 2)

Question

I try to run this code (binary classification) but still stuck with this error: ValueError: Error when checking target: expected avg_pool to have 4 dimensions, but got array with shape (100, 2)

    NUM_CLASSES = 2
    CHANNELS = 3
    IMAGE_RESIZE = 224
    RESNET50_POOLING_AVERAGE = 'avg'
    DENSE_LAYER_ACTIVATION = 'softmax'
    OBJECTIVE_FUNCTION = 'binary_crossentropy'
    NUM_EPOCHS = 10
    EARLY_STOP_PATIENCE = 3
    STEPS_PER_EPOCH_VALIDATION = 10
    BATCH_SIZE_TRAINING = 100
    BATCH_SIZE_VALIDATION = 100
    resnet_weights_path = '../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'
    train_data_dir = "C:\\Users\\Desktop\\RESNET"
    model = ResNet50(include_top=True, weights='imagenet')
    x = model.get_layer('avg_pool').output
    predictions = Dense(1, activation='sigmoid')(x)
    model = Model(input = model.input, output = predictions)
    print(model.summary())
    model.layers.pop() 
    model = Model(input=model.input,output=model.layers[-1].output)
    sgd = optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
    model.compile(loss='binary_crossentropy', optimizer=SGD(lr=0.01, momentum=0.9), metrics=   ['binary_accuracy'])
    data_dir = "C:\\Users\\Desktop\\RESNET"
    batch_size = 32
    from keras.applications.resnet50 import preprocess_input
    from keras.preprocessing.image import ImageDataGenerator
    image_size = IMAGE_RESIZE
    data_generator = ImageDataGenerator(preprocessing_function=preprocess_input)
    def append_ext(fn):
        return fn+".jpg"
    dir_path = os.path.dirname(os.path.realpath(__file__))
    train_dir_path = dir_path + '\data'
    onlyfiles = [f for f in listdir(dir_path) if isfile(join(dir_path, f))]
    NUM_CLASSES = 2
    data_labels = [0, 1]
    t = []
    maxi = 25145
    LieOffset = 15799
    i = 0
    while i < maxi: # t = tuple
    if i <= LieOffset:
        t.append(label['Lie'])
    else:
        t.append(label['Truth'])
    i = i+1
    train_datagenerator = ImageDataGenerator(rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True,
        validation_split=0.2) 
    train_generator = train_datagenerator.flow_from_directory(
        train_data_dir,
        target_size=(image_size, image_size), 
        batch_size=BATCH_SIZE_TRAINING,
        class_mode='categorical', shuffle=False, subset='training') # set as training data

    validation_generator = train_datagenerator.flow_from_directory(
        train_data_dir, # same directory as training data 
        target_size=(image_size, image_size), 
        batch_size=BATCH_SIZE_TRAINING,
        class_mode='categorical', shuffle=False, subset='validation') 

    (BATCH_SIZE_TRAINING, len(train_generator), BATCH_SIZE_VALIDATION, len(validation_generator))
    from sklearn.grid_search import ParameterGrid
    param_grid = {'epochs': [5, 10, 15], 'steps_per_epoch' : [10, 20, 50]}
    grid = ParameterGrid(param_grid)
# Accumulate history of all permutations (may be for viewing trend) and keep watching for lowest   val_loss as final model
    for params in grid:
    fit_history = model.fit_generator(
        train_generator,
        steps_per_epoch=STEPS_PER_EPOCH_TRAINING,
        epochs = NUM_EPOCHS,
        validation_data=validation_generator,
        validation_steps=STEPS_PER_EPOCH_VALIDATION,
        callbacks=[cb_checkpointer, cb_early_stopper]
    )
    model.load_weights("../working/best.hdf5")

Answer 1

Remove these lines and it'll work,

model.layers.pop() 
model = Model(input=model.input,output=model.layers[-1].output)

The former is removing the last( Dense ) layer, and the letter means create a model without the last( Flatten , as Dense is already popped) layer.

It doesn't make sense as your target data is (100, 2). Why do you put them there in the first place?

Also I think this line

predictions = Dense(1, activation='sigmoid')(x)

Will error, as your target data is 2 channel, if it does error then change this to

predictions = Dense(2, activation='sigmoid')(x)

Update

The output of avg_pool is 4 dimention, (batch_size, height, width, channel). You need to do the Flatten first or use GlobalAveragePooling2D instead of AveragePooling2D .

Like

x = model.get_layer('avg_pool').output
x = keras.layers.Flatten()(x)
predictions = Dense(1, activation='sigmoid')(x)

Or

model = ResNet50(include_top=False, pooling='avg', weights='imagenet')  # `pooling='avg'` makes the `ResNet50` include a `GlobalAveragePoiling` layer and `include_top=False` means that you don't include the imagenet's output layer
x = model.output  # as I use `include_top=False`, you don't need to care the layer name, just use the model's output right away
predictions = Dense(1, activation='sigmoid')(x)

Also just as @bit01 said, change class_mode='categorical' to class_mode='binary'.

Answer 2

Change class_mode='categorical' to class_mode='binary' in both train_generator and validation_generator .

Additionally, delete the following lines as you have already created model.

model.layers.pop() 
model = Model(input=model.input,output=model.layers[-1].output)

So, your model would be like:

model = ResNet50(include_top=True, weights='imagenet')
x = model.get_layer('avg_pool').output
x = Flatten()(x)
predictions = Dense(1, activation='sigmoid')(x)
model = Model(input = model.input, output = predictions)