将 Keras model 与 Tensorflow 数据集拟合

Question

I'm reading Aurélien Géron's book , and on chapter 13, I'm trying to use Tensorflow datasets (rather than Numpy arrays) to train Keras models.

1. The dataset

The dataset comes from sklearn.datasets.fetch_california_housing , which I've exported to CSV. The first few lines look like this:

MedInc,HouseAge,AveRooms,AveBedrms,Population,AveOccup,Latitude,Longitude,MedHouseVal
3.3083,20.0,5.387832699619772,1.0,853.0,3.2433460076045626,37.53,-120.79,1.083
2.1932,29.0,5.164444444444444,1.1288888888888888,726.0,3.2266666666666666,37.53,-120.8,0.906
1.875,15.0,5.327102803738318,1.1495327102803738,189.0,1.766355140186916,37.53,-120.81,1.813

As you can see there's 8 features and 1 target ( MedHouseVal , on the last column).

I reimport it using the following code:

def parse_csv_line(line: bytes) -> Tuple[tf.Tensor, tf.Tensor]:
    parsed = tf.io.decode_csv(line, record_defaults=[0.]*9)
    return parsed[:-1], parsed[-1:]

def load_dataset_csv(paths: Union[str, List[str]]) -> tf.Dataset:
    return tf.data.Dataset.list_files(paths)                        \
        .interleave(lambda p: tf.data.TextLineDataset(p).skip(1))   \
        .map(parse_csv_line)                                        \
        .prefetch(1)

train_set = load_dataset_csv("./housing.train.csv")
val_set = load_dataset_csv("./housing.val.csv")
test_set = load_dataset_csv("./housing.test.csv")

So far so good:

>>> list(test_set.take(1))
[(<tf.Tensor: shape=(8,), dtype=float32, numpy=
  array([   3.3083   ,   20.       ,    5.3878326,    1.       ,
          853.       ,    3.243346 ,   37.53     , -120.79     ],
        dtype=float32)>,
  <tf.Tensor: shape=(1,), dtype=float32, numpy=array([1.083], dtype=float32)>)]

2. Model, first try

Then I define my model:

model = keras.models.Sequential(
    [
        keras.layers.Dense(30, input_shape=(8,)),
        keras.layers.BatchNormalization(),
        keras.layers.Dense(1),
    ],
)
model.compile(loss="mean_squared_error")

model.fit(
    train_set,
    epochs=1,  # debugging
    validation_data=val_set,
)

but I get the following error:

ValueError: Input 0 of layer sequential_63 is incompatible with the layer: expected axis -1 of input shape to have value 8 but received input with shape (8, 1)

3. Model, second try

If I set the input shape to (8, 1) instead of (8,) , then I get this warning

WARNING:tensorflow:Model was constructed with shape (None, 8, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 8, 1), dtype=tf.float32, name='input_15'), name='input_15', description="created by layer 'input_15'"), but it was called on an input with incompatible shape (8, 1, 1).

When I try to make a prediction, I get this weird result:

>>> model.predict(test_set.take(1))

WARNING:tensorflow:Model was constructed with shape (None, 8, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 8, 1), dtype=tf.float32, name='dense_159_input'), name='dense_159_input', description="created by layer 'dense_159_input'"), but it was called on an input with incompatible shape (8, 1, 1).
WARNING:tensorflow:6 out of the last 174 calls to <function Model.make_predict_function.<locals>.predict_function at 0x169f02b80> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
array([[[2.592609 ]],
       [[2.591457 ]],
       [[2.5924652]],
       [[2.592768 ]],
       [[2.533997 ]],
       [[2.5926137]],
       [[2.590248 ]],
       [[2.601169 ]]], dtype=float32)

I understand that Tensorflow is displeased with the shape of the tensors, still, why is it giving me 8 predictions?

4. Model, third try

At this point I'm not really sure what to do, so I tried adding a Flatten layer:

model = keras.models.Sequential(
    [
        keras.layers.Flatten(),
        keras.layers.Dense(30, input_shape=(8,)),
        keras.layers.BatchNormalization(),
        keras.layers.Dense(1),
    ],
)

This time I get no warning or error, but when I try to make a prediction, I still get 8 results:

>>> model.predict(test_set.take(1))

array([[2.5953178],
       [2.5949838],
       [2.5952766],
       [2.5953639],
       [2.5783124],
       [2.5953195],
       [2.594633 ],
       [2.5978017]], dtype=float32)

I don't really understand what I'm doing wrong. Trying to train the first model (with input shape (8,) and no Flatten ) with Numpy arrays instead works just fine.

Any help would be really appreciated.

Thanks in advance!

Answer 1

Just as the official docs for tf.keras.Sequential suggest, no batch_size needs to be provided when inputs are instances of tf.data.Dataset while calling tf.keras.Sequential.fit() ,

Integer or None. Number of samples per gradient update. If unspecified, batch_size will default to 32. Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

In case of tf.data.Dataset , the fit() method expects a batched dataset.

To batch the tf.data.Dataset , use the batch() method,

batched_ds = ds.batch( batch_size )

Thus the dataset will now provide batches of data ( of shape ( batch_size, 8 ) ) instead of the entire data ie of shape ( num_samples, 8 ) .

Tip:

To unbatch a dataset ie to reshape the data to ( num_samples, 8 again, use the unbatch() method provided.