How to pass validation_data to Model.fit + Dataset?

Question

I am trying to train a simple Sequential network on generated data. I have a precomputed validation dataset.

To feed the inputs, I am using tf.data.Dataset API like suggested here: https://stackoverflow.com/a/48134765/231238

var train = Dataset.from_tensor_slices(ValueTuple.Create(trainInputs, trainOutputs));
train = train
    .repeat(2000000)
    .shuffle(buffer_size: 1024 * 8 * InterpolateBy)
    .batch(1024);
model.fit_dyn(train,
    epochs: 6*1024,
    steps_per_epoch: 4
    // line below does not work:
    , validation_data: (testInputs, testOutputs)
);

It works fine without validation_data .

If I pass validation_data as a tuple of tensors, like in the example above, eg (testInputs, testOutputs) , it throws 'TypeError : float() argument must be a string or a number, not 'NoneType' . (This is what I used to do with train data too before switching to Dataset , and validation worked)

If I wrap testInputs and testOutputs into a Dataset similarly to the train data, eg Dataset.from_tensor_slices(ValueTuple.Create(testInputs, testOutputs))

I get a different error: ValueError : Error when checking input: expected sequential_input to have 2 dimensions, but got array with shape (347,) .

Here 347 is the size of feature vector, hence testInputs.shape is (221, 347) and testOutputs.shape is (221, 1)

Answer 1

From our discussion, we can clarify some of the things.

First off, not very sure about the error when directly feeding it as a tuple. Might be needing more information about the data for it.

As far as feeding validation with tf data, when we use from_tensor_slices , "we create a dataset whose elements are slices of given tensors". With respect to this example, the input we are feeding is a tuple with respective shapes (221,347) and (221,1). What from_tensor_slices does is that it slices the respective numpy arrays along the 0th dimension (which is of size 221 here). The method will thus create a dataset, where each element is a tuple of shape (347,) and (1,) respectively. There will be 221 such elements in the dataset.

If we use the from_tensors method on the other hand, it creates a dataset with a single element, which comprise of the given tensors as input. So it is equivalent to directly feeding the numpy data as it is through a dataset object.

Here is a brief example of how this works for a much smaller dimension:

import numpy as np
import tensorflow as tf
np.random.seed(42)
example_train = np.random.randn(4, 4)
example_test = np.random.randn(4, 1)

print("Example Train:", example_train)
print("Example Test:", example_test)

dataset1 = tf.data.Dataset.from_tensor_slices((example_train, example_test))
dataset2 = tf.data.Dataset.from_tensors((example_train, example_test))

it1 = dataset1.make_one_shot_iterator().get_next()
it2 = dataset2.make_one_shot_iterator().get_next()

with tf.Session() as sess:
    for i in range(4):
        print("Element {} of dataset1: {}".format(i,sess.run([it1])))
    print ("Element 0 of dataset2: ", sess.run([it2]))

Result:

Example Train: [[ 0.49671415 -0.1382643   0.64768854  1.52302986]
 [-0.23415337 -0.23413696  1.57921282  0.76743473]
 [-0.46947439  0.54256004 -0.46341769 -0.46572975]
 [ 0.24196227 -1.91328024 -1.72491783 -0.56228753]]
Example Test: [[-1.01283112]
 [ 0.31424733]
 [-0.90802408]
 [-1.4123037 ]]
Element 0 of dataset1: [(array([ 0.49671415, -0.1382643 ,  0.64768854,  1.52302986]), array([-1.01283112]))]
Element 1 of dataset1: [(array([-0.23415337, -0.23413696,  1.57921282,  0.76743473]), array([0.31424733]))]
Element 2 of dataset1: [(array([-0.46947439,  0.54256004, -0.46341769, -0.46572975]), array([-0.90802408]))]
Element 3 of dataset1: [(array([ 0.24196227, -1.91328024, -1.72491783, -0.56228753]), array([-1.4123037]))]
Element 0 of dataset2:  [(array([[ 0.49671415, -0.1382643 ,  0.64768854,  1.52302986],
       [-0.23415337, -0.23413696,  1.57921282,  0.76743473],
       [-0.46947439,  0.54256004, -0.46341769, -0.46572975],
       [ 0.24196227, -1.91328024, -1.72491783, -0.56228753]]), array([[-1.01283112],
       [ 0.31424733],
       [-0.90802408],
       [-1.4123037 ]]))]

Regarding my comments about the batch method, by setting the batch_size to be 221 for putting things back together, if we change the dataset1 code to something like this and modify our printing to something like this for example:

dataset1 = tf.data.Dataset.from_tensor_slices((example_train, example_test)).batch(4)

with tf.Session() as sess:
    print ("Element 0 of dataset1: ", sess.run([it1]))    

Our result:

Element 0 of dataset1:  [(array([[ 0.49671415, -0.1382643 ,  0.64768854,  1.52302986],
       [-0.23415337, -0.23413696,  1.57921282,  0.76743473],
       [-0.46947439,  0.54256004, -0.46341769, -0.46572975],
       [ 0.24196227, -1.91328024, -1.72491783, -0.56228753]]), array([[-1.01283112],
       [ 0.31424733],
       [-0.90802408],
       [-1.4123037 ]]))]

which you can see is the same as using from_tensors.