Tensorflow Keras 損失為 NaN

Question

正如您在下面看到的，我嘗試使用 tensorflow/keras 創建一個 MLP。 但不幸的是，擬合時損失始終為 NaN。 你有什么建議嗎？

作為第二條錯誤消息，我在嘗試使用model.score測量准確性時收到消息“'Functional' object has no attribute 'score'”，但我認為這是由第一個觸發的問題。

謝謝大家

import numpy as np
import matplotlib.pyplot as plt

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.decomposition import PCA
from mpl_toolkits import mplot3d
from sklearn import datasets
from various import printShapes, printNumpy, print_Model_Accuracy, printLARGE, checkFormat
from sklearn.datasets import make_blobs
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

np.random.seed(1234)
#%matplotlib qt 
#%matplotlib inline
plt.rcParams["figure.figsize"] = [4*2, 4*2]

if 0:
    iris = datasets.load_iris()
    X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.80, random_state=1234)

if 1:
    X, y = make_blobs(n_features=4, centers=3, n_samples=1000, cluster_std = 5.0,  random_state=1234)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=1234)
    
print ("Target Label Example: y_train[0]:")
print (y_train[0])
print (type(y_train[0]))

printLARGE("MLP classifier TENSORFLOW")

tf.random.set_seed(1234)

Epochs = 10

inputs = keras.Input(shape=(4,), name="digits")
x = layers.Dense(100, activation="tanh", name="dense_1")(inputs)
x = layers.Dense(4, activation="tanh", name="dense_2")(x)
outputs = layers.Dense(3, activation="softmax", name="predictions")(x)

model = keras.Model(inputs=inputs, outputs=outputs)

model.compile(
    optimizer=keras.optimizers.RMSprop(),  # Optimizer
    loss=keras.losses.SparseCategoricalCrossentropy(), # Loss function to minimize
    metrics=[keras.metrics.SparseCategoricalAccuracy()],  # List of metrics to monitor
)
printShapes(X_train, "X_train", y_train, "y_train")
# TRAINING      
model.fit(X_train, y_train, batch_size=64, epochs=Epochs)
printShapes(X_test, "X_test", y_test, "y_test")
# INFERENCE
y_test_predproba = model.predict(X_test)
print(y_test_predproba)
y_test_pred = np.argmax(y_test_predproba, axis = 1)
print(y_test_pred)

print_Model_Accuracy(model, X_test, y_test, y_test_pred)

Answer 1

1. 在隱藏層中使用 tanh 激活函數沒有任何意義。 應該是 ReLU。
2. 使用更多的隱藏層會比在第一層使用更多的單元更好。 [為你的任務]
3. 然而，使用更多的隱藏層會使模型更容易過擬合，添加Dropout層可以解決這個問題。

最后，你的模型應該是，

inputs = keras.Input(shape=(4,), name="digits")
x = layers.Dense(32, activation="relu", name="dense_1")(inputs)
x = layers.Dropout(0.2)(x)
x = layers.Dense(24, activation="relu", name="dense_2")(x)
x = layers.Dropout(0.2)(x)
x = layers.Dense(16, activation="relu", name="dense_2")(x)
outputs = layers.Dense(3, activation="softmax", name="predictions")(x)

model = keras.Model(inputs=inputs, outputs=outputs)