获取最终预测掩码中的轮廓坐标

Question

I'm implementing a Unet model for nuclei segmentation. The model is working fine and the segmentation was successfully done. However, I want to save the contours on a json file to properly load it in a web app.

Here's my original image:

And here's the corresponding predicted mask.

I tried to use findContours on the mask but overlapped cells would be recognized as one. Note that overlapped cells got that green boundary to differentiate nucleis.

What I want is to get the coordinates of the single nuclei contours and save it as a json, like this:

{"1_0.jpeg-1":{"filename":"1_0.jpeg","size":-1,"regions":[{"shape_attributes":{"name":"polyline","all_points_x":[
    216.0,
    510.5,
    215.5,
    510.0,
    216.0,
    509.5,
    216.5,
    510.0,
    216.0,
    510.5
],"all_points_y":[]},"region_attributes":{}}],"file_attributes":{}}}

This is my predict function where I save the mask of each image-to-predict

if __name__ == '__main__':
    t0 = timeit.default_timer()
    args_models = ['best_resnet101_2_fold0.h5']
    weights = [os.path.join(args.models_dir, m) for m in args_models]
    models = []
    for w in weights:
        model = make_model(args.network, (None, None, 3))
        print("Building model {} from weights {} ".format(args.network, w))
        model.load_weights(w)
        models.append(model)
    os.makedirs(test_pred, exist_ok=True)
    print('Predicting test')
    for d in tqdm(listdir(test_folder)):
        final_mask = None
        for scale in range(1):
            fid = d
            print(path.join(test_folder, '{0}'.format(fid)))
            img = cv2.imread(path.join(test_folder, '{0}'.format(fid)), cv2.IMREAD_COLOR)[...,::-1]

            if final_mask is None:
                final_mask = np.zeros((img.shape[0], img.shape[1], OUT_CHANNELS))
            if scale == 1:
                img = cv2.resize(img, None, fx=0.75, fy=0.75, interpolation=cv2.INTER_AREA)
            elif scale == 2:
                img = cv2.resize(img, None, fx=1.25, fy=1.25, interpolation=cv2.INTER_CUBIC)

            x0 = 16
            y0 = 16
            x1 = 16
            y1 = 16
            if (img.shape[1] % 32) != 0:
                x0 = int((32 - img.shape[1] % 32) / 2)
                x1 = (32 - img.shape[1] % 32) - x0
                x0 += 16
                x1 += 16
            if (img.shape[0] % 32) != 0:
                y0 = int((32 - img.shape[0] % 32) / 2)
                y1 = (32 - img.shape[0] % 32) - y0
                y0 += 16
                y1 += 16
            img0 = np.pad(img, ((y0, y1), (x0, x1), (0, 0)), 'symmetric')

            inp0 = []
            inp1 = []
            for flip in range(2):
                for rot in range(4):
                    if flip > 0:
                        img = img0[::-1, ...]
                    else:
                        img = img0
                    if rot % 2 == 0:
                        inp0.append(np.rot90(img, k=rot))
                    else:
                        inp1.append(np.rot90(img, k=rot))

            inp0 = np.asarray(inp0)
            inp0 = preprocess_inputs(np.array(inp0, "float32"))
            inp1 = np.asarray(inp1)
            inp1 = preprocess_inputs(np.array(inp1, "float32"))

            mask = np.zeros((img0.shape[0], img0.shape[1], OUT_CHANNELS))

            for model in models:
                pred0 = model.predict(inp0, batch_size=1)
                pred1 = model.predict(inp1, batch_size=1)
                j = -1
                for flip in range(2):
                    for rot in range(4):
                        j += 1
                        if rot % 2 == 0:
                            pr = np.rot90(pred0[int(j / 2)], k=(4 - rot))
                        else:
                            pr = np.rot90(pred1[int(j / 2)], k=(4 - rot))
                        if flip > 0:
                            pr = pr[::-1, ...]
                        mask += pr  # [..., :2]

            mask /= (8 * len(models))
            mask = mask[y0:mask.shape[0] - y1, x0:mask.shape[1] - x1, ...]
            if scale > 0:
                mask = cv2.resize(mask, (final_mask.shape[1], final_mask.shape[0]))
            final_mask += mask
        final_mask /= 1
        if OUT_CHANNELS == 2:
            final_mask = np.concatenate([final_mask, np.zeros_like(final_mask)[..., 0:1]], axis=-1)
        final_mask = final_mask * 255
        final_mask = final_mask.astype('uint8')
        cv2.imwrite(path.join(test_pred, '{0}'.format(fid)), final_mask, [cv2.IMWRITE_PNG_COMPRESSION, 9])

    elapsed = timeit.default_timer() - t0
    print('Time: {:.3f} min'.format(elapsed / 60))

Do you have any idea how to get the coordinates of each classified nuclei? The json part should be easy but I don't get how can I get the countours' coordinates. Should I do it after the predicted mask is written? Or should I do it on my predict process?

Kind Regards

Answer 1

Coordinates points of a contour can be found by

mask = np.zeros(imgray.shape,np.uint8)
cv.drawContours(mask,[cnt],0,255,-1)
pixelpoints = np.transpose(np.nonzero(mask))

Have a look at https://docs.opencv.org/3.4/d1/d32/tutorial_py_contour_properties.html

[EDIT]

To separate the cells, you can remove green boundary by extracting the blue color alone.

I take the predicted mask image as input.

img = cv2.imread('1.jpg')

hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# define range of blue color in HSV
lower_blue = np.array([110,50,50])
upper_blue = np.array([130,255,255])
# Threshold the HSV image to get only blue colors
mask = cv2.inRange(hsv, lower_blue, upper_blue)
blue_only = cv2.bitwise_and(img,img, mask= mask)

im2, contours, hierarchy = cv2.findContours(mask,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)

#draw contours with indexes and save coordinates to a txt file
with open('coords.txt', 'w+') as f:
    for i,cnt in enumerate(contours):
        cv2.drawContours(blue_only, cnt, -1, (0,0,255), 1)
        cv2.putText(blue_only, str(i), (cnt[0][0][0], cnt[0][0][1]),cv2.FONT_HERSHEY_SIMPLEX, 1,(0,0,255), 1)
        f.writelines("contour " + str(i) +" :" + str(cnt))  

cv2.imshow('img',img)
cv2.imshow('mask',mask)
cv2.imshow('blue_only',blue_only)