Tensorflow dataset.from_generator()：如何检查加载了哪些数据？

Question

I am creating a tensorflow dataset (in tensorflow 1.15) using a generator that loads images from a folder.

I want to know what images I am using, so I print their filename when I load one.

A simplified version of the generator that I use is the following:

def gen():
    for i in itertools.count(0):
        data = np.load(list_of_file_names[i])
        print(f'image_name[{i}]: {list_of_file_names[i]}')
        yield tf.convert_to_tensor(data)

Where list_of_file_names is the sorted list of .npy files in a certain folder: image0.npy, image1.npy, image2.npy, image3.npy, image4.npy, image5.npy, image6.npy, image7.npy ...

This is a simplified version of the input_fn function that I use for my estimator.predict() :

def input_fn():
    dataset = tf.data.Dataset.from_generator(gen, (tf.float32))
    dataset = dataset.batch(batch_size, drop_remainder=True)
    return dataset.make_one_shot_iterator().get_next()

This is the result from the print statement inside the generator:

image_name[0]: image0.npy
image_name[1]: image1.npy
image_name[2]: image2.npy
image_name[3]: image3.npy
2022-12-12 15:48:10.276336: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
image_name[4]: image4.npy
image_name[5]: image5.npy

I apply a batch size of 3 to my dataset.

When I process a batch of data, the result is computed on a batch made of the files relative to image4.npy , image5.npy and image6.npy (also contained in the folder).

Why is the name of image6.npy not printed?

Why are the names of the other (previous) files printed?

Answer 1

image4,image5, image6 correspond to image3.npy,image4.npy, image5.npy respectively as the index starts from 0.

Your image6 file name is image5.npy