i am developing a system for general-purpose audio tagging using keras.
I have the following data input: x_train has 10 different data for each input (data_leng,max,min,etc) and y_train represents 41 possible labels (guitar,bass,etc)
x_train shape = (7104, 10)
y_train shape = (41,)
print(x_train[0])
[ 3.75732000e+05 -2.23437546e-05 -1.17187500e-02 1.30615234e-02
2.65964586e-03 2.65973969e-03 9.80024859e-02 1.13624850e+00
1.00003528e+00 -1.11458333e+00]
print(y_train[0])
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]
My model is:
from keras.models import Sequential
from keras.optimizers import SGD
from keras.layers import Dense, Dropout, Activation
model = Sequential()
model.add(Dense(units=128, activation='relu', input_dim=10))
model.add(Dropout(0.5))
model.add(Dense(units=64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(units=32, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(41, activation='softmax'))
opt = SGD(lr=0.0001, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
model.fit(np.array(x_train), np.array(y_train), epochs=5, batch_size=8)
This is my result:
Epoch 1/5
7104/7104 [==============================] - 1s 179us/step - loss: 15.7392 - acc: 0.0235
Epoch 2/5
7104/7104 [==============================] - 1s 132us/step - loss: 15.7369 - acc: 0.0236
Epoch 3/5
7104/7104 [==============================] - 1s 133us/step - loss: 15.7415 - acc: 0.0234
Epoch 4/5
7104/7104 [==============================] - 1s 132us/step - loss: 15.7262 - acc: 0.0242
Epoch 5/5
7104/7104 [==============================] - 1s 132us/step - loss: 15.6484 - acc: 0.0291
As you can see, my results show very high data loss and very low accuracy but the main problem is when i try to predict the result, cause for each one input the output is the same. How can i fix this ?
pre = model.predict(np.array(x_train), batch_size=8, verbose=0)
for i in pre:
print(i)
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
...
In your Dense layers, you need to specify the Input_dim only for the first layer.
Keras take care of the Dim in the other layers.
So try :
model = Sequential()
model.add(Dense(units=128, activation='relu', input_dim=10))
model.add(Dropout(0.5))
model.add(Dense(units=64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(units=32, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(41, activation='softmax'))
And maybe your regularization is too strong for this kind of data, try a dropout less strong or no dropout at all.
Last thing you can do is increasing your learning rate, start with something like 1e-3 and see if something change.
Hope i helped you
You can try testing others optimizers and try do change your last layer activation. I already haved the same problem, i was using Softmax activation in last Dense layer, i changed to Sigmoid and works well.
A good strategy are modifying the model's architecture, add more layers, change dropout values, etc...
Hope i helped you. Good luck!
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.