相同的NN架构在张量流和keras中提供不同的精度

Question

A neural network trained on iris dataset using [4, 4] hidden layers and created separately in tensorflow and keras gives different results.

While the tensorflow model gives 96.6 % accuracy on test, keras model gives only around 50%. The various hyper parameters like learning rate, optimiser, mini batch size, etc were the same in both cases.

Keras model

model = Sequential()

model.add(Dense(units = 4, activation = 'relu', input_dim = 4))
model.add(Dropout(0.25))
model.add(Dense(units = 4, activation = 'relu'))
model.add(Dropout(0.25))
model.add(Dense(units = 3, activation = 'softmax'))

adam = Adam(epsilon = 10**(-6), lr = 0.01)
model.compile(optimizer = 'adagrad', loss = 'categorical_crossentropy', metrics = ['accuracy'])

one_hot_labels = keras.utils.to_categorical(y_train, num_classes = 3)

model.fit(X_train, one_hot_labels, epochs = 50, batch_size = 40)

Tensorflow model

feature_columns = [tf.feature_column.numeric_column(key = name,
                                                   shape = (1),
                                                   dtype = tf.float32) for name in list(X_train.columns)]

classifier = tf.estimator.DNNClassifier(hidden_units = [4, 4],
                                       feature_columns = feature_columns,
                                       n_classes = 3,
                                       dropout = 0.25,
                                       model_dir = './DNN_model')

train_input_fn = tf.estimator.inputs.pandas_input_fn(x = X_train,
                                              y = y_train,
                                              batch_size = 40,
                                              num_epochs = 50,
                                              shuffle = False)

classifier.train(input_fn = train_input_fn, steps = None)

For the keras model, I did try changing the learning rate, increasing the number of epochs, using different optimisers, etc. As such, the accuracy remained poor. Clearly, both the models are doing different things, but on the surface, they seem identical to me for all the key aspects.

Any help is appreciated.

Answer 1

they have the same architecture , and that's all.

The difference in performance is coming from one or more of these factors:

• You have Dropout. Therefore your networks in every start behaving differently (check how the Dropout works);

• Weight initializations, which method you're using in Keras and TensorFlow?

• Check all parameters of the optimizer.