Tensorflow iterator fails to iterate

Question

I am working on a project related to instance segmentation. I am trying to train a SegNet with my own image dataset which comprises a set of images and their corresponding masks, and I have successfully used tf.Dataset to load my data. But every time I use the feedable iterator to feed the dataset to SegNet, my program is always terminated without any error or warning. My code is shown below.

load_satellite_image() is used to read filename for images and dataset() is used to load images with tf.Dataset. It seems that the iterator fails to update the input pipeline.

     train_path = "data_example/train.txt"
     val_path = "data_example/test.txt"

     config_file = 'config.json'
     with open(config_file) as f:
         config = json.load(f)

     train_img, train_mask = load_satellite_image(train_path)   
     val_img, val_mask = load_satellite_image(val_path)   

     train_dataset = dataset(train_img, train_mask, config, True, 0, 1)  
     val_dataset = dataset(val_img, val_mask, config, True, 0, 1)  

     train_iter = train_dataset.make_initializable_iterator()
     validation_iter = val_dataset.make_initializable_iterator()

     handle = tf.placeholder(tf.string, shape=[])    

     iterator =  tf.data.Iterator.from_string_handle(handle,
                 train_dataset.output_types,train_dataset.output_shapes)

     next_element = iterator.get_next()

     with tf.Session() as Sess:
         sess.run(train_iter.initializer)
         sess.run(validation_iter.initializer)
         train_iter_handle = sess.run(train_iter.string_handle())
         val_iter_handle = sess.run(validation_iter.string_handle())

         for i in range(2):
             print("1")
             try:
                 while True:

                     for i in range(5):
                         print(sess.run(next_element,feed_dict={handle:train_iter_handle}))
                         print('----------------------------','\n')

                     for i in range(2):
                         print(sess.run(next_element,feed_dict={handle:val_iter_handle}))
             except tf.errors.OutOfRangeError:
                     pass

After running the code above, I got:

         In [2]: runfile('D:/python_code/tensorflow_study/SegNet/load_data.py', 
         wdir='D:/python_code/tensorflow_study/SegNet')

        (tf.float32, tf.int32)
        (TensorShape([Dimension(360), Dimension(480), Dimension(3)]), TensorShape([Dimension(360), 
        Dimension(480), Dimension(1)]))
        (tf.float32, tf.int32)
        (TensorShape([Dimension(360), Dimension(480), Dimension(3)]), TensorShape([Dimension(360), 
        Dimension(480), Dimension(1)]))
        WARNING:tensorflow:From D:\Anaconda\envs\tensorflow-gpu\lib\site- 
        packages\tensorflow\python\data\ops\dataset_ops.py:1419: colocate_with (from 
        tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
        Instructions for updating:
        Colocations handled automatically by placer.

        In [1]:

I am confused that my code is terminated without any reason. As you can see, I can get the shape and datatype of training/ validation images and masks, which means the problem has nothing to do with my dataset. However, the for loop in the tf.Session() is not executed and I cannot get the result of print("1") . The iterator is not executed by sess.run() as well. Anyone have met this problem before?

Thanks!!!

Answer 1

Problem solved. It's a stupid mistake that wastes me a lot of time. The reason why my program is terminated without error message is that I am using stupid Spyder to write my code, and I don't know why it doesn't show the error message. Actually, there exists an error message produced by TensorFlow. By coincidence, I ran my code via the command window of Anaconda and I got this error message:

2020-04-30 17:31:03.591207: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at whole_file_read_ops.cc:114 : Invalid argument: NewRandomAccessFile failed to Create/Open: D:\Study\PhD\python_code\tensorflow_study\SegNet\data_example\trainannot\ges_517405_679839_21.jpg

The iterator doesn't work because Tensorflow cannot find mask locations. The image and mask locations are stored in a text file like this:

data_example\train\ges_517404_679750_21.jpg,data_example\trainannot\ges_517404_679750_21.jpg
data_example\train\ges_517411_679762_21.jpg,data_example\trainannot\ges_517411_679762_21.jpg

The left side is the locations of raw images and the right side is the locations of their masks. In the beginning, I used split(",") to get the location of images and masks separately, but it seems that there is something wrong with the locations of masks. So I checked the code that is used to generate the text file:

file.writelines([Train_path[i],',',TrainAnnot_path[i],'\n'])

Each line in the text file ends with \n , and this is why Tensorflow cannot get the location of the masks. So I replaced file.writelines([Train_path[i],',',TrainAnnot_path[i],'\n']) with file.writelines([Train_path[i],' ',TrainAnnot_path[i],'\n']) , and used strip().split(" ") rather than split(" ") . That solves the problem.