Cannot convert tf.keras.preprocessing.image_dataset_from_directory to np.array

Question

I am trying to create a image classification model using CNN. For that I am reading the data using the tf.keras.preprocessing.image_dataset_from_directory function.

This is the code:

train_ds = tf.keras.preprocessing.image_dataset_from_directory(data_dir_train,seed=123,validation_split = 0.2,subset = 'training',image_size=(img_height, img_width),batch_size=batch_size)

Then I am trying to convert the dataset in np.array object. My code is

x_train = np.array(train_ds)

But when I print x_train , I am getting

array(<BatchDataset shapes: ((None, 180, 180, 3), (None,)), types: (tf.float32, tf.int32)>, dtype=object)

The object train_ds is of shape (2000,180,180,3) . I am not sure what is wrong in my code.

Answer 1

When using image_dataset_from_directory :

... image_dataset_from_directory(main_directory, labels='inferred') will return a tf.data.Dataset that yields batches of images from the subdirectories...

One option to get the data you want, is to use take to create a Dataset with at most count elements from this dataset.

import tensorflow as tf
import numpy as np

img = np.empty((6000,180,180,3), dtype=np.float32)
label = np.empty((6000,), dtype=np.int32)

train_ds = tf.data.Dataset.from_tensor_slices((img,label)).batch(2000)
print(train_ds) # <BatchDataset shapes: ((None, 180, 180, 3), (None,)), types: (tf.float32, tf.int32)>

for batch in train_ds.take(1):
    img_batch, label_batch = batch
    print(img_batch.shape) # (2000, 180, 180, 3)
    print(label_batch.shape) # (2000,)

Depending on how you're structuring your code, you may not even need to convert to a numpy.array , since tf.keras.Model fit accepts tf.data datasets.

model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)