了解如何将边界框坐标与图像匹配

Question

After many attempts I'm able to match the bounding box of Tensorflow tfjs Coco-SSD with the actual image of my phone camera, but I really don't understand how I did it, let me explain:

The output of the model looks like this: bbox: [x, y, width, height]

I discovered just by brute force that if I multiply each value for 2.5 everything match perfectly. But I have no idea where that 2.5 comes from, and I would suspect that may not work in phones with other dimensions, so if anyone could help me to understand this it would be very helpful.

Thanks!

My code looks like this:

<TouchableOpacity style={{...styles.box, borderColor: 'red', 
            left: p.bbox[0]*2.5, // p.bbox[0], 
            top: p.bbox[1]*2.5,
            width: p.bbox[2]*2.5, 
            height: p.bbox[3]*2.5}}>              
            </TouchableOpacity>

Answer 1

what you're passing to tfjs is an image data in some resolution (is it video or image or canvas, doesn't matter - it all comes down to image data at the end).

then module resizes it to whatever resolution model is trained on.

and model results are relative to that resolution, not to your input. so you need to take a look at whatever is actual model resolution and rescale results accordingly.