Error in input pipeline using queues in tensorflow

Question

I am getting this error for the below mentioned code. Kindly help me out with this. This code is printing variables number of batches everytime I run this code. I am unable to figure out the error.

OutOfRangeError (see above for traceback): FIFOQueue '_2_batch/fifo_queue' is closed and has insufficient elements (requested 15, current size 0) [[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/fifo_queue, batch/n)]]

import tensorflow as tf 
import numpy as np 
import os

batch_size = 16 
min_queue = 256


def load_image():
    length = calculate_size("./Coco_subset_5356")

    names = tf.train.match_filenames_once("./Coco_subset_5356/*.jpg")

    # Extracted names from the file  
    filename_queue = tf.train.string_input_producer(names)

    #Initialised the File reader
    reader = tf.WholeFileReader()

    key, value = reader.read(filename_queue)
    my_img = tf.image.decode_jpeg(value, channels = 3)
    my_img = tf.image.resize_images(my_img, [256,256])
    my_img.set_shape((256,256,3))


    print(length)

    images = tf.train.batch(
      [my_img],
      batch_size=batch_size,
      num_threads=1,
      capacity=min_queue + 3*batch_size)

     print(images)

with tf.Session() as sess:  
    #sess.run(init_op)  
    tf.local_variables_initializer().run()
    #print(tf.get_collection(tf.GraphKeys.LOCAL_VARIABLES))

    #For coordination between queue runner and the reader
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord = coord)
    for j in range(length/batch_size):
        #optimizer.run(feed_dict={input_batch: images[i])
        x = sess.run(images)    
        print(j)
        print(x.shape)

    coord.request_stop()
    coord.join(threads)

def calculate_size(img_dir):
    file = []
    for subdir, dirs, files in os.walk(img_dir):
        for i in files:
            i = os.path.join(img_dir, i)
            file.append(i)
        length = len(file)
    return length

load_image()

Answer 1

Disclaimer: This is not a proper answer, but I can't post code in a comment and this might point you in the right direction.

As anticipated in my comment to the question, I had the very same error on one of my datasets. In my case, the problem was the fact that not all images in my dataset could be decoded as jpg files . Truncated images and, in my case, 1x1 pixel images were hidden in my data and whenever I'd try to prcess the dataset I'd have the queues closing reporting errors I had a hard time fiding.

I solved the problem writing a small filter script that goes through all files one-by-one and logs those it cannot process:

import tensorflow as tf
import glob

image_data_placeholder = tf.placeholder(tf.string)

filenames = glob.glob(r'C:\my_test_folder\*.jpg')
decoded_img_op = tf.image.decode_jpeg(image_data_placeholder)

with tf.Session() as sess:
  for fn in filenames:
    with open(fn, 'rb') as fp:
      image_data = fp.read()
    try:
      sess.run(decoded_img_op, feed_dict={image_data_placeholder:image_data})
    except:
      # log the filename o the broken image (or delete/move/etc)
      print('Cannot decode file {}'.format(fn))

It's not the most efficient of implementations, but it was good enough for my use-case.