使我的数据适合Keras顺序模型和密集层并产生输出

Question

I have structured data that looks like this.

faults.head()

Fault   DEALER  FAILMODE    FAILCODEMODE    DAYS UNTIL FAILURE  TERRITORY CODE  DESIGN PHASE CODE   PLANT ID CODE
0   CAMPAIGN/TRP    31057   CAMPAIGN    BNRBC1  283.0   102 62  82
1   INTERMITTENT PROBL  24126   SPECIAL (NO FAILURE)    XXIPNF  126.0   102 62  82
2   DSID #DSBCG2058 TAG #362783 EXHAUST SYSTEM. U...    0   CLOGGED, PLUGGED WITH FOREIGN MATERIAL, DIRT/D...   USDVDR  118.0   102 62  82
3   INTERMITTENT PROBL  20943   SPECIAL (NO FAILURE)    XXIPNF  97.0    102 62  82
4   CAMPAIGN    19134   CAMPAIGN    USSCR1  315.0   102 62  82

I'm trying to predict the class FAILMODE. There's only 122 unique values in FAILMODE. Those are my classes.

Based on all the other data in the rows, I want to have a one-hot matrix or even the class itself be a product of the computation on my test set. Here's my code so far-

from keras.models import Sequential
from keras.layers import Dense
Using Theano backend.

faults_testing = faults[:14843]
faults_training = faults[14844:]

model = Sequential()
model.add(Dense(len(faults.FAILMODE.unique()) + 20, input_dim=len(faults_training), init='uniform', activation='relu'))
model.add(Dense(len(faults_training), init='uniform', activation='relu'))
model.add(Dense(len(faults.FAILMODE.unique()), init='uniform', activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

Here is where the tutorial says-

model.fit(X, Y, nb_epoch=len(faults_training), batch_size=10)

I don't know what X or Y is so I just tried the following-

model.fit(faults_training['FAILMODE'], faults_testing['FAILMODE'], nb_epoch=len(faults_training), batch_size=10)

It resulted in this error-

ValueError                                Traceback (most recent call last)
<ipython-input-54-e8765933cfb9> in <module>()
----> 1 model.fit(faults_training['FAILMODE'], faults_testing['FAILMODE'], nb_epoch=len(faults_training), batch_size=10)

ValueError: Error when checking model input: expected dense_input_1 to have shape (None, 34631) but got array with shape (34631L, 1L)

Please be thorough with your answer. Thank you!

Answer 1

Regular neural network (including Keras Sequential model) only accepts float for data (X) and int or one-hot-encoding for label/class (Y). So you need to convert your dataset to match the requirements. So what you may want to do:

1. Map all categorical (string) values (eg. CAMPAIGN/TRP, BNRBC1, XXIPNF) to float (it will be better if you can normalize the data)
2. Put all the data columns (excluding the label column) in X
3. Put the label column (should be 1 column only) inside Y and convert to one-hot-encoding using to_categorical eg Y = to_categorical(Y)
4. Split the training and testing data using function such as train_test_split X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.33)
5. Train the model using model.fit(X_train, Y_train, nb_epoch=100, batch_size=10) . Adjust the nb_epoch and batch_size later on based on your expectation of speed and accuracy of the training
6. Evaluate the accuracy using scores = model.evaluate(self, X_test, Y_test, batch_size=10)

You can also check this article to get idea how to convert categorical data to numeric http://fastml.com/converting-categorical-data-into-numbers-with-pandas-and-scikit-learn/