Keras分段故障（核心转储）

Question

I am trying to train a convolutional neural network on Keras 1.2.2 on top of Theano 0.8.2 (python 2.7). I am able to import keras and theano without getting an error.

The error only occurs after circa 2-5 minutes after running the following code.

#Prepare images etc.

model = Sequential()

model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1],
                        border_mode='valid',
                        input_shape=input_shape))
model.add(Activation('relu'))
model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1]))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.25))

model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1]))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.25))

model.add(Convolution2D(nb_filters * 2, kernel_size[0], kernel_size[1]))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(128))
model.add(Dense(64))
model.add(Dense(nb_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adadelta',
              metrics=['accuracy'])

model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,
          verbose=1, validation_data=(X_test, Y_test))

score = model.evaluate(X_test, Y_test, verbose=0)
print('Test score:', score[0])
print('Test accuracy:', score[1])

Which creates following output:

X_train shape: (984, 1, 1000, 1000)
984 train samples
246 test samples
Train on 984 samples, validate on 246 samples
Epoch 1/4
[1]+  Segmentation fault      (core dumped)

So it seems like the model was compiled and training has started. I was able to train a model before with smaller images (shape: (400, 1, 500, 500). Could changing the number of training images and their size lead to the error? I also tried updating Keras to 2.0 and Theano to the current dev version but it didn't help.

Any suggestions?

Answer 1

If you run K.clearsession() incorrectly, you may get a segmentation fault. This is because each neural network graph is held in a default session, when you clear the session, the graph gets removed from memory. Putting K.clearsession() is smart to do at the top of a code (esp if you're using jupyter) because it will clear any graphs held in memory.

If you have this in your code, try removing it! Good Luck!