如何在PyTorch中加载具有多个JSON注释的图像

Question

I would like to know how I can use the data loader in PyTorch for the custom file structure of mine. I have gone through PyTorch documentation, but all those are with separate folders with class.

My folder structure consists of 2 folders(called training and validation), each with 2 subfolders(called images and json_annotations). Each image in the "images" folder has multiple objects(like cars, cycles, man etc) and each is annotated and have separate JSON files. Standard coco annotation is followed. My intention is to make a neural network which can do real-time classification from videos.

Edit 1: I have done the coding as suggested by Fábio Perez.

class lDataSet(data.Dataset):
    def __init__(self, path_to_imgs, path_to_json):
        self.path_to_imgs = path_to_imgs
        self.path_to_json = path_to_json
        self.img_ids = os.listdir(path_to_imgs)

    def __getitem__(self, idx):
        img_id = self.img_ids[idx]
        img_id = os.path.splitext(img_id)[0]
        img = cv2.imread(os.path.join(self.path_to_imgs, img_id + ".jpg"))
        load_json = json.load(open(os.path.join(self.path_to_json, img_id + ".json")))
        #n = len(load_json)
        #bboxes = load_json['annotation'][n]['segmentation']
        return img, load_json

    def __len__(self):
        return len(self.image_ids)

When I try this

l_data = lDataSet(path_to_imgs = '/home/training/images', path_to_json = '/home/training/json_annotations')

I'm getting l_data with l_data[][0] - images and l_data with json. Now I'm confused. How will I use it with finetuning example availalbe in PyTorch? In that example, dataset and dataloader is done as shown below.
https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html

# Create training and validation datasets
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x]) for x in ['train', 'val']}
# Create training and validation dataloaders
dataloaders_dict = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=batch_size, shuffle=True, num_workers=4) for x in ['train', 'val']}

Answer 1

You should be able to implement your own dataset with data.Dataset . You just need to implement __len__ and __getitem__ methods.

In your case, you can iterate through all images in the image folder (then you can store the image ids in a list in your Dataset ). Then, you use the index passed to __getitem__ to get the corresponding image id. With this image id, you can read the corresponding JSON file and return the target data that you need.

Something like this:

class YourDataLoader(data.Dataset):
    def __init__(self, path_to_imgs, path_to_json):
        self.path_to_imags = path_to_imgs
        self.path_to_json = path_to_json
        self.image_ids = iterate_through_images(path_to_images)

    def __getitem__(self, idx):
        img_id = self.image_ids[idx]
        img = load_image(os.path.join(self.path_to_images, img_id)
        bboxes = load_bboxes(os.path.join(self.path_to_json, img_id)
        return img, bboxes

    def __len__(self):
        return len(self.image_ids)

In iterate_through_images you get all the ids (eg filenames) of images in a directory. In load_bboxes you read the JSON and get the information you need.

I have a JSON loader implementation here if you want a reference.