简体繁体 English

3D 标注用于实例分割

[英]3D annotation for instance segmentation

原文 2020-07-27 20:41:15 1 2 deep-learning/ computer-vision/ annotations/ image-segmentation

I'm trying to annotate some data for 3D instance segmentation.我正在尝试为 3D 实例分割注释一些数据。 While it's fairly straightforward to draw masks for each 2D plane, it's not obvious how to connect the same "instances" together post-annotation (ie. connect the "red" masks together, connect the "blue" masks together) without laboriously making sure the instances are instance-matched (ie. colour-coded to make sure "red" masks always connect with "red" masks).虽然为每个 2D 平面绘制蒙版相当简单，但如何在注释后将相同的“实例”连接在一起（即，将“红色”蒙版连接在一起，将“蓝色”蒙版连接在一起）而不费力地确保实例是实例匹配的（即颜色编码以确保“红色”掩码始终与“红色”掩码连接）。

A naive approach I have thought of is to make many 2D segmentation masks, and calculate the center of mass for each object detected.我想到的一种天真的方法是制作许多 2D 分割掩码，并为检测到的每个 object 计算质心。 I can later re-assign the instances based on the closest matching center of mass, but I worry this would inadvertently generate "crossed-over" segmentation instances (illustrated below).稍后我可以根据最接近的匹配质心重新分配实例，但我担心这会无意中生成“交叉”分割实例（如下图所示）。 What are some high-throughput strategies to generate 3D annotations?生成 3D 注释的高通量策略有哪些？

2 个解决方案

The boundary of your 2-d slices could be used as constraints to obtain the optimal 3-d surface, as proposed in 1 .您的 2-d 切片的边界可用作获得最佳 3-d 表面的约束，如1 中所建议的那样。

However, I think it is easier to generate 3-d labels from markers, such as 2 .但是，我认为从标记生成 3-d 标签更容易，例如2 。 Its implementation is available in here (Fill free open an issue if you encounter any problems:P).它的实现可在此处获得（如果遇到任何问题，请免费填写打开问题：P）。

Also, the napari package could be useful to develop the GUI without much effort.此外， napari package 可用于轻松开发 GUI。

[1] Grady, Leo. [1] 格雷迪，里奥。 "Minimal surfaces extend shortest path segmentation methods to 3D." “最小曲面将最短路径分割方法扩展到 3D。” IEEE Transactions on Pattern Analysis and Machine Intelligence 32.2 (2008): 321-334. IEEE 模式分析和机器智能汇刊 32.2 (2008): 321-334。

[2] Falcão, Alexandre X., and Felipe PG Bergo. [2] Falcão、Alexandre X. 和 Felipe PG Bergo。 "Interactive volume segmentation with differential image foresting transforms." “具有差分图像森林变换的交互式体积分割。” IEEE Transactions on Medical Imaging 23.9 (2004): 1100-1108. IEEE 医学影像汇刊 23.9 (2004): 1100-1108。

You can use 3D Slicer's Segment Editor .您可以使用3D Slicer 的 Segment Editor 。 It is free, open-source, has many built-in tools, and customizable/extensible in Python or C++ (you can plug in your own segmentation method with minimal effort).它是免费的、开源的、有许多内置工具，并且可以在 Python 或 C++ 中自定义/扩展（您可以轻松插入自己的分段方法）。 To solve a segmentation task, typically you first figure out a good segmentation workflow (what tools to use, in what combination and what parameters) using interactive GUI, then if necessary you can make it semi-automatic or fully automatic using Python scripting.要解决分割任务，通常您首先使用交互式 GUI 找出一个好的分割工作流程（使用什么工具、组合和参数），然后如果需要，您可以使用 Python 脚本使其半自动或全自动。

You can create a segmentation by contouring every image slice, but it would be too tedious.您可以通过对每个图像切片进行轮廓处理来创建分割，但这太乏味了。 Instead, you can use 3D region growing (Grow from seeds effect) or segment on just a few slices and interpolate between them (Fill between slices effect).相反，您可以使用 3D 区域生长（从种子效果中生长）或仅在几个切片上进行分段并在它们之间进行插值（在切片之间填充效果）。