使用 SYNTHIA 数据集进行实例分割

Question

I haved used SYNTHIA dataset (RAND-CITYSCAPES subset) for semantic segmentation using the first channel of the label image as suggested in the previous post:

How to read the label(annotation) file from Synthia Dataset? .

In the README file of the dataset, it says the second channel is devoted for the instance segmentation:

"GT/LABELS: folder containing png files (one per image). Annotations are given in two channels. The first channel contains the class of that pixel (see the table below). The second channel contains the unique ID of the instance for those objects that are dynamic (cars, pedestrians, etc.)."

Then, I try to read the instance and semantic segmentation maps using the following code I wrote:

def read_synthia_label(path):
    raw_label = np.asarray(imageio.imread(path, format='PNG-FI'))
    seg_label = Image.fromarray(np.uint8(raw_label[:,:,0]))
    inst_label = Image.fromarray(np.uint16(raw_label[:,:,1])) 
    return seg_label, inst_label

However, when I check the consistency of the labels, I observed that segmentation labels for the same instance label within the same image are not the same. Ie the following assert throws error:

with PathManager.open(instance_id_file, "rb") as f:
    inst_image = np.asarray(Image.open(f), order="F")

with PathManager.open(segmentation_file, "rb") as f:
    segm_image = np.asarray(Image.open(f), order="F")

flattened_inst_ids = np.unique(inst_image)

for instance_id in flattened_inst_ids:
    inds_for_same_inst = np.where(instance_id == inst_image.ravel())[0]
    assert(segm_image.ravel()[inds_for_same_inst[0]] == segm_image.ravel()[inds_for_same_inst[-1]]) # throws error   

Am I reading the instance segmentation in the wrong way? Had anyone ever used SYNTHIA for instance segmentation? I could not find any documentation on this on the web, so any help would be appreciated.

Answer 1

I experienced the same behavior you described in SYNTHIA VIDEO SEQUENCES.

However, this is an issue just for some classes. For dynamic objects, as stated in README (see below), the segmentation labels for the same instance label within the same image are the same, ie, consistent.

README:

• GT/LABELS: folder containing png files (one per image). Annotations are given in two channels. The first channel contains the class of that pixel (see the table below). The second channel contains the unique ID of the instance for those objects that are dynamic ( cars, pedestrians, etc. ).