How to load images and labels seperately in a dataset loaded by tensorflow_datasets

Question

import tensorflow_datasets as tfds

train_ds = tfds.load('cifar100', split='train[:90%]').shuffle(1024).batch(32)
val_ds = tfds.load('cifar100', split='train[-10%:]').shuffle(1024).batch(32)

I want to convert train_ds and val_ds into something like this: x_train, y_train and x_val, y_val (x for images and y for labels). The Keras API uses train and test data split (this seems to be the case in sklearn too), but I do not want to use any test data at all here.

I have tried this, but it didn't work (and I do understand why this doesn't work, but I don't know how else can I convert my training data to images and labels):

x_train = train_ds['image']

# TypeError: 'BatchDataset' object is not subscriptable

Answer 1

Not the best way, I created lists firstly to inspect them. I think you want something like:

train_ds = tfds.load('mnist', split='train[:90%]')

train_examples_labels = tfds.as_numpy(train_ds)

x_train = []
y_train = []


for features_labels in train_examples_labels:
    x_train.append(features_labels['image'])
    y_train.append(features_labels['label'])

features_labels is a dictionary here:

features_labels.keys()
dict_keys(['image', 'label'])

After you can convert them into numpy arrays.

x_train = np.array(x_train, dtype = 'float32')
y_train = np.array(y_train, dtype = 'float32')

Answer 2

I found a better solution:

train_ds, val_ds = tfds.load(name="cifar100", split=('train[:90%]','train[-10%:]'), batch_size=-1, as_supervised=True)

x_train, y_train = tfds.as_numpy(train_data)
x_val, y_val = tfds.as_numpy(val_data)