Keras总是预测相同的输出

Question

Keras will always predict the same class for every input i give hm. There are currently four classes. News, Weather, Sport and Economy.

The training set consists of a lot of different texts, where the class is the same as its topic. There are a lot more texts classified as News and Sport, than there are texts for Weather and Economy.

• News: 12112 texts
• Weather: 1685 texts
• Sport: 13669 texts
• economy: 1282 texts

I would have expected the model to be biased towards Sport and News, but instead it is completely biased towards Weather with every input beeing classified as Weather with at least 80% confidence.

Just to add to my confusion: While training the annotator will reach accuracy scores from 95% to 100%(sic!). I guess I am doing something really stupid here but I don't know what it is.

This one is how i call my classifier. It runs on python 3 on a Windows pc.

with open('model.json') as json_data:
model_JSON = json.load(json_data)

model_JSON = json.dumps(model_JSON) 
model = model_from_json(model_JSON)

model.load_weights('weights.h5')

text = str(text.decode())   
encoded = one_hot(text, max_words, split=" ")

tokenizer = Tokenizer(num_words=max_words)
matrix = tokenizer.sequences_to_matrix([encoded], mode='binary')

result = model.predict(matrix)

legende = ["News", "Wetter", "Sport", "Wirtschaft"]
print(str(legende))
print(str(result))

cat = numpy.argmax(result)  
return str(legende[cat]).encode()

This one is how I train my classifier. I omitted the part where I fetch the data from a database. This is done on a Linux VM. I already tried changing the loss and activation around, but nothing happened. Also I am curently trying to use more epochs but up to now that hasn't helped either.

max_words = 10000
batch_size=32
epochs=15

rows = cursor.fetchall()

X = []
Y = []

# Einlesen der Rows
for row in rows:
    X.append(row[5])
    Y.append(row[1])

num_classes = len(set(Y))
Y = one_hot("$".join(Y), num_classes, split="$")


for i in range(len(X)):
    X[i] = one_hot(str(X[i]), max_words, split=" ")

split = round(len(X) * 0.2)     

x_test = np.asarray(X[0:int(split)])
y_test = np.asarray(Y[0:int(split)])

x_train = np.asarray(X[int(split):len(X)])
y_train = np.asarray(Y[int(split):len(X)])

print('x_test shape', x_test.shape)
print('y_test shape', y_test.shape)

print(num_classes, 'classes')

#vektorisieren
tokenizer = Tokenizer(num_words=max_words)
x_train = tokenizer.sequences_to_matrix(x_train, mode='binary')
x_test = tokenizer.sequences_to_matrix(x_test, mode='binary')

#klassenvektor zu binärer klassenmatrix
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

#model erstellen
model = Sequential()

model.add(Dense(512, input_shape=(max_words,)))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
    optimizer='adam',
    metrics=['accuracy'])


history = model.fit(x_train, y_train,
    batch_size=batch_size,
    epochs=epochs,
    verbose=1,
    validation_split=0.1
    )

score = model.evaluate(x_test, y_test,
    batch_size=batch_size, 
    verbose=1
    )

print('Test score', score[0])
print('Test accuracy', score[1])

#write model to json
print("writing model to json")
model_json = model.to_json()
with open("model.json", 'w') as json_file:
    json_file.write(model_json)

#save weights as hdf5
print("saving weights to hdf5")
model.save_weights("weights.h5")

Answer 1

Thanks to the tip that @Daniel Möller gave me I found out what the problem was. His tip was to look at how many instances of each Class are contained in your training set.

In my case I found out, that hashing your classes with One_Hot is not smart, as it will sometimes encode multiple classes with the same number. For me One_Hot encoded nearly everything as a 1. This way Keras learned to only predict 1.