Why am I getting different results on a prediction using the same Keras model and input?

Question

posting here is my last resort cause I can't find anything like it online. I trained a model to classify embeddings into categories (a simple three layer Dense neural network).

Now I want to use the trained model to make predictions in real time, but I discovered that if I input the whole test dataframe to the model, get the prediction for say the element number i , and compare it to the prediction that I get by inputting just the element number i of the test data frame into the model, I get different results. This is the code in case I didn't explain it good enough:

i = 522
y_pred = model.predict(X_test)
y_pred_2 = model.predict(X_test.iloc[[i]])

print (f'{np.argmax(y_pred[i])} {np.argmax(y_pred_2)}')

output: 8 5

It's like my model is behaving differently if it processes the whole test set in a single run than if it processes a single row at a time. I'm using pandas for the input data.

EDIT : More info, the output shapes of y_pred and y_pred_2 are (603, 10) and (1, 10) respectively, where 10 is the number of classes I have.

Some example values for both predictions, with an arbitrary i :

y_pred[i]: array([1.3353945e-02, 2.8374636e-09, 1.4435661e-08, 3.4135045e-18,
   7.7986561e-02, 3.7737598e-03, 2.0284578e-10, 2.7154891e-03,
   9.0203673e-01, 1.3346069e-04], dtype=float32)

y_pred_2 = array([[1.1702824e-16, 1.6781385e-37, 2.5281618e-33, 0.0000000e+00,
        2.3075200e-09, 1.0000000e+00, 9.9125501e-35, 6.2606384e-22,
        5.8689110e-14, 2.3486194e-24]], dtype=float32)

Answer 1

np.argmax defaults to the row axis. You're getting the maximum prediction across rows, and you want it across columns. Try:

print(f'{np.argmax(y_pred[i], axis=1)} {np.argmax(y_pred_2, axis=1)}')