Mac OS; Python3; FileNotFoundError: [Errno 2] No such file or directory: ''. How to fix it?

Question

I'm quit new at Python, and tried to launch the script

convert_to_tfrecord.py

(Neural Networks; It should train dataset of Images with some libraries ...)

Instruction:

Now you're ready to run the TFRecord script. Run the command below from the tensorflow/models/research directory, and pass it the following flags (run it twice: once for training data, once for test data):

python convert_labels_to_tfrecords.py \
--output_path=train.record \ 
--images_dir=path/to/your/training/images/ \
--labels_dir=path/to/training/label/xml/

So to suit my OS X, I ran this script by python3... changes name of script... and set a directory ... I'm at directory where my script is; Where my folder is; Where my libraries are. So in my case:

python3 convert_to_tfrecord.py \
--output_path=train.record \ 
--images_dir=ENFj/ \
--labels_dir=ENFj/xml/

Result:

Oleksandrs-MacBook-Air:research jaskier$ python3 convert_to_tfrecord.py \
> --output_path=train.record \ 
Traceback (most recent call last):
  File "convert_to_tfrecord.py", line 89, in <module>
    tf.app.run()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "convert_to_tfrecord.py", line 81, in main
    for filename in os.listdir(FLAGS.images_dir):
FileNotFoundError: [Errno 2] No such file or directory: ''
Oleksandrs-MacBook-Air:research jaskier$ --images_dir=ENFj/ \
> --labels_dir=ENFj/xml/

Code of "convert_to_tfrecord":

import os
import io
import xml.etree.ElementTree as ET
import tensorflow as tf

from object_detection.utils import dataset_util
from PIL import Image


flags = tf.app.flags
flags.DEFINE_string('output_path', '', 'Path to output TFRecord')
flags.DEFINE_string('images_dir', '', 'Path to directory of images')
flags.DEFINE_string('labels_dir', '', 'Path to directory of labels')
FLAGS = flags.FLAGS


def create_tf_example(example):

    image_path = os.getcwd() + '/' +  FLAGS.images_dir + example
    labels_path = os.getcwd() + '/' +  FLAGS.labels_dir + os.path.splitext(example)[0] + '.xml'

    # Read the image
    img = Image.open(image_path)
    width, height = img.size
    img_bytes = io.BytesIO()
    img.save(img_bytes, format=img.format)

    height = height
    width = width
    encoded_image_data = img_bytes.getvalue()
    image_format = img.format.encode('utf-8')

    # Read the label XML
    tree = ET.parse(labels_path)
    root = tree.getroot()
    xmins = xmaxs = ymins = ymaxs = list()

    for coordinate in root.find('object').iter('bndbox'):
        xmins = [int(coordinate.find('xmin').text)]
        xmaxs = [int(coordinate.find('xmax').text)]
        ymins = [int(coordinate.find('ymin').text)]
        ymaxs = [int(coordinate.find('ymax').text)]

    classes_text = ['tswift'.encode('utf-8')]
    classes = [1]

    tf_example = tf.train.Example(features=tf.train.Features(feature={
        'image/height': dataset_util.int64_feature(height),
        'image/width': dataset_util.int64_feature(width),
        'image/filename': dataset_util.bytes_feature(encoded_image_data),
        'image/source_id': dataset_util.bytes_feature(encoded_image_data),
        'image/encoded': dataset_util.bytes_feature(encoded_image_data),
        'image/format': dataset_util.bytes_feature(image_format),
        'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
        'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
        'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
        'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
        'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
        'image/object/class/label': dataset_util.int64_list_feature(classes),
    }))
    return tf_example


def main(_):
    writer = tf.python_io.TFRecordWriter(FLAGS.output_path)

    for filename in os.listdir(FLAGS.images_dir):
        tf_example = create_tf_example(filename)
        writer.write(tf_example.SerializeToString())

    writer.close()


if __name__ == '__main__':
    tf.app.run()

Tried different stuff: 1. UPDATED request to (one line + deleted "\\" because as someone mentioned below, it was an error of page's PHP interpreter... it used \\n and forgot to hide the output text):

python3 convert_to_tfrecord.py 
--output_path=train.record 
--images_dir=ENFj
--labels_dir=ENFj/xml

2."find . -name ".DS_Store" -delete" 3. Tried again:

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/PIL/Image.py", line 2543, in open fp = builtins.open(filename, "rb") IsADirectoryError: [Errno 21] Is a directory: '/Users/jaskier/Downloads/models/research/ENFj/xml'

Answer 1

The code is set up to interpret all files in --images_dir to some image processing machinery. This means that any non-image resources in the --images_dir will cause the script to break.

One solution is to ensure --images_dir contains only image files (ie ensure that directory doesn't contain XML files or files that begin with a ., like .git or .DS_Store .

Another solution would be to modify the source code itself to only work on image files. Something like this could be used:

import glob

# only match jpg files in the images_dir
for filename in glob.glob(FLAGS.images_dir + '/*.jpg'):
  tf_example = create_tf_example(filename)
  # copy the other lines here as needed

Answer 2

In my case it was just a simple dumb mistake: when I changed manually the name of my folder:

I have left a SPACE after the name - and when I was calling it with os or glob.glob I was getting: FileNotFoundError: [Errno 2] No such file or directory:

After spending some time searching for the problem(even changing some files names and removing some subfolders) I went back to the folder, deleted that extra space, and everything worked just fine.