如何从图像的热图数据生成边界框数据？

Question

I have a group of images and some separate heatmap data which (imperfectly) explains where subject of the image is. The heatmap data is in a numpy array with shape (224,224,3). I would like to generate bounding box data from this heatmap data.

The heatmaps are not always perfect, So I guess I'm wondering if anyone can think of an intelligent way to do this.

Here are some examples of what happens when I apply the heatmap data to the image:

I found a solution to this in matlab, but I have no idea how to read this code! I am a python programmer, unfortunately.https://github.com/metalbubble/CAM/tree/master/bboxgenerator

Anyone have any ideas about how to approach something like this?

Answer 1

I am not quite sure how the heatmap data of your project exactly looks like, but it seems to me that you can use something like Selective Search . You can also have a look on this interesting paper . Maybe you can use this approach on your dataset.

Answer 2

I'm attempting a similar method for automating the creation of bounding boxes (since, lets face it: creating boxes manually takes along time)

this other stackpost covers a similar idea: EDIT: (i originally had put a link to the current stack post 🤦 - but here is the stack post i was referring to) Generating bounding boxes from heatmap data

the problem at hand that i recognize is that heatmaps can be fragmented and a bit arbitrary. the solution that comes to mind initially is setting a threshold of the heat map. So in the case of the example heat map images - when applying a bounding box cover all regions that are yellow/orange/red than say green/blue.

Answer 3

It depends on how many bounding boxes you need. You can set a threshold and have multiple bounding boxes for each of the highly activated regions, or try connecting the regions (by a morphological operation maybe) and calculate a single bounding box for connected activated pixels.