tf.keras给出nan丢失和非验证错误

Question

I'm trying to program Deep neural Network using tf.keras API, I feel my model is right I have removed all the nan values but I'm still getting NAN values during training. The Data set I have been using is Wiscon Cancer dataset from UCI here

My Code:

from tensorflow import keras
import pandas as pd
import tensorflow as tf

df = pd.read_csv('breastc.csv.csv')
df.dropna()
id_ = df['ID'].tolist()
del df['ID']
labels = df['Class'].tolist()
import numpy as np
del df['Class']
column_list='Compactness'
df[column_list] = df[column_list].apply(pd.to_numeric, errors='coerce')

model = keras.Sequential()
model.add(keras.layers.Dense(64,activation='relu',input_shape = (9,)))
model.add(keras.layers.Dense(64,activation='relu'))
model.add(keras.layers.Dense(1,activation='softmax'))

model.summary()

X=df.iloc[:].values

model.compile(optimizer=tf.train.AdamOptimizer(0.01),
              loss='mse',       # mean squared error
              metrics=['mae'])
model.fit(X,labels,batch_size=32,epochs=10,validation_split=0.2)

After the fit statement I'm getting the following results

 Train on 559 samples, validate on 140 samples Epoch 1/10 559/559 [==============================] - 0s 599us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 2/10 559/559 [==============================] - 0s 82us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 3/10 559/559 [==============================] - 0s 86us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 4/10 559/559 [==============================] - 0s 84us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 5/10 559/559 [==============================] - 0s 87us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 6/10 559/559 [==============================] - 0s 83us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 7/10 559/559 [==============================] - 0s 80us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 8/10 559/559 [==============================] - 0s 77us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 9/10 559/559 [==============================] - 0s 73us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan Epoch 10/10 559/559 [==============================] - 0s 62us/step - loss: nan - mean_absolute_error: nan - val_loss: nan - val_mean_absolute_error: nan <tensorflow.python.keras._impl.keras.callbacks.History at 0x15c94a80cc0> 

As we can see there is no training happening. Please guide me.

Yours Sincerely, Vidit Shah

Answer 1

when you have a classification problem with 2 or more classes and you want to choose only one of those you should usually put the last layer to have the number of output neurons equal to the number of classes and set its activation function to softmax (so you will have a distribution over the classes as output). Once you get the output distribution you can consider as prediction the index in the output vector (ie the class) that received the highest probability.

Another thing you should change is the loss function, when you use a softmax as output activation function you need to use the crossentropy loss that measure the distance between two distributions (in this case the output from your network and the gold distribution - all 0s and a 1 at the index that correspond to the correct class). In case you have the gold label represented as indices of the correct class (0 or 1 in your case) you can use those indices directly by setting sparse_corssentropy as loss function (it will automatically transform your indices in one-hot vectors).

To wrap everything up, you can transform your code as follow:

from tensorflow import keras
import pandas as pd
import tensorflow as tf

df = pd.read_csv('breastc.csv.csv')
df.dropna()
id_ = df['ID'].tolist()
del df['ID']
labels = df['Class'].tolist()
import numpy as np
del df['Class']
column_list='Compactness'
df[column_list] = df[column_list].apply(pd.to_numeric, errors='coerce')

model = keras.Sequential()
model.add(keras.layers.Dense(64,activation='relu',input_shape = (9,)))
model.add(keras.layers.Dense(64,activation='relu'))
model.add(keras.layers.Dense(2,activation='softmax'))

model.summary()

X=df.iloc[:].values

model.compile(optimizer=tf.train.AdamOptimizer(0.01),
              loss='categorical_crossentropy',
              metrics=['accuracy'])
model.fit(X,labels,batch_size=32,epochs=10,validation_split=0.2)