Tensorflow / Keras predict function output length does not match input length

Question

I am using Ubuntu 19.04 (Disco Dingo), Python 3.7.3, and TensorFlow 1.14.0.

I noticed that the number of outputs given by the tensorflow.keras.Sequential.predict function is different than the number of inputs. Furthermore, it appears that there is no relation between the inputs and outputs.

Example:

import tensorflow as tf
import math
import numpy as np
import json

# We will train the model to recognize an XOR

x = [ [0,0], [0,1], [1,0], [1,1] ]

y = [ 0, 1, 1, 0 ]

xt = tf.cast(x, tf.float64)
yt = tf.cast(y, tf.float64)

# This model should be more than enough to learn an XOR

L0 = tf.keras.layers.Dense(2)
L1 = tf.keras.layers.Dense(4, activation=tf.nn.relu)
L2 = tf.keras.layers.Dense(4, activation=tf.nn.relu)
L3 = tf.keras.layers.Dense(2, activation=tf.nn.softmax)

model = tf.keras.Sequential([L0,L1,L2,L3])

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

model.fit(
    x=xt,
    y=yt,
    batch_size=32,
    epochs=1000, # Try to overfit data
    shuffle=False,
    steps_per_epoch=math.ceil(len(x)/32)
)

# While it is training, the loss drops to near zero
# and the accuracy goes to 100%.
# The large number of epochs and the small number of training examples
# should mean that the network is overtrained.

print("testing")

for i in range(len(y)):
    m = tf.cast([x[i]],tf.float64)
    # m should be the ith training example
    values = model.predict(m,steps=1)
    best = np.argmax(values[0])
    print(x[i],y[i],best)

The output I always get is:

(input, correct answer, predicted answer)

[0, 0] 0 0
[0, 1] 1 0
[1, 0] 1 0
[1, 1] 0 0

or

[0, 0] 0 1
[0, 1] 1 1
[1, 0] 1 1
[1, 1] 0 1

So, even though I thought that the network would be overtrained, even though the program said that the accuracy was 100% and the loss was virtually zero, the output looks as though the network hadn't trained at all.

Stranger yet is when I replace the testing section with the following:

print("testing")

m = tf.cast([], tf.float64)
values = model.predict(m, steps=1)
print(values)

I would think that this would return an empty array or throw an exception. Instead it gives:

[[0.9979249  0.00207507]
 [0.10981816 0.89018184]
 [0.10981816 0.89018184]
 [0.9932179  0.0067821 ]]

This corresponds to [0,1,1,0]

So even though it was given nothing to predict on, it still gives out predictions for something. And it appears as though the predictions match up with what what we would expect from sending the entire training set into the predict method.

Replacing the testing section again:

print("testing")

m = tf.cast([[0,0]], tf.float64)
# [0,0] is the first training example
# the output should be something close to [[1.0,0.0]]
values = model.predict(m, steps=1)
for j in range(len(values)):
    print(values[j])
exit()

I get:

[0.9112452  0.08875483]
[0.00552484 0.9944752 ]
[0.00555605 0.99444395]
[0.9112452  0.08875483]

This corresponds to [0,1,1,0]

So asking it to predict on zero inputs, gives out 4 predictions. Asking it to predict on one input gives out 4 predictions. Furthermore, the predictions it gives out looks like what we would expect if we put the entire training set into the predict function.

Any ideas as to what's going on? How do I get my network to give exactly one prediction for each input given?

Answer 1

Providing the solution here (Answer Section), even though it is present in the Comment Section, for the benefit of the community.

Upgrading Tensorflow from 1.14.0 >=2.0 has resolved the issue.

After upgrading test section works as expected

m = tf.cast([[0,0]], tf.float64)
# [0,0] is the first training example
# the output should be something close to [[1.0,0.0]]
values = model.predict(m, steps=1)
for j in range(len(values)):
    print(values[j])
exit()

Output:

[0.9921625  0.00783745]