Faster-RCNN，我们为什么不只使用RPN进行检测？

Question

As we know, faster-RCNN has two main parts: one is region proposal network(RPN), and another one is fast-RCNN.

My question is, now that region proposal network(RPN) can output class scores and bounding boxes and is trainable, why do we need Fast-RCNN?

Am I thinking it right that the RPN is enough for detection (red circle), and Fast-RCNN is now becoming redundant (blue circle)?

Answer 1

Short answer: no they are not redundant. The R-CNN article and its variants popularized the use of what we used to call a cascade. Back then for detection it was fairly common to use different detectors often very similar in structures to do detection because of their complementary power.

If the detections are partly orthogonal it allows to remove false positive along the way.

Furthermore by definition both parts of R-CNN have different roles the first one is used to discriminate objects from background and the second one to discriminate fine grained categories of objects from themselves (and from the background also).

But you are right if there is only 1 class vs the background one could use only the RPN part to to detection but even in that case it would probably better the result to chain two different classifiers (or not see eg this article )

PS: I answered because I wanted to but this question is definitely unsuited for stackoverflow

Answer 2

faster-rcnn is a two-stage method comparing to one stage method like yolo, ssd, the reason faster-rcnn is accurate is because of its two stage architecture where the RPN is the first stage for proposal generation and the second classification and localisation stage learn more precise results based on the coarse grained result from RPN.

So yes, you can, but your performance is not good enough

Answer 3

If you just add a class head to the RPN Network, you would indeed get detections, with scores and class estimates.

However, the second stage is used mainly to obtain more accurate detection boxes.

Faster-RCNN is a two-stage detector, like Fast R-CNN. There, Selective Search was used to generate rough estimates of the location of objects and the second stage then refines them, or rejects them.

Now why is this necessary for the RPN? So why are they only rough estimates?

One reason is the limited receptive field: The input image is transformed via a CNN into a feature map with limited spatial resolution. For each position on the feature map, the RPN heads estimate if the features at that position correspond to an object and the heads regress the detection box. The box regression is done based on the final feature map of the CNN. In particular, it may happen that the correct bounding box on the image is larger than the corresponding receptive field due to the CNN.

Example: Lets say we have an image depicting a person and the features at one position of the feature map indicate a high possibiliy for the person. Now, if the corresponding receptive field contains only the body parts, the regressor has to estimate a box enclosing the entire person, although it "sees" only the body part.

Therefore, RPN creates a rough estimate of the bounding box. The second stage of Faster RCNN uses all features contained in the predicted bounding box and can correct the estimate.

In the example, RPN creates a too large bounding box, which is enclosing the person (since it cannot the see the pose of the person), and the second stage uses all information of this box to reshape it such that it is tight. This however can be done much more accurate, since more content of the object is accessable for the network.

Answer 4

我认为蓝色圆圈是完全多余的，只需添加一个类分类层（为每个包含对象的边界框提供类）应该可以正常工作，这就是单次检测器在精度受损的情况下所做的。

Answer 5

根据我的理解，RPN 仅用于二进制检查 bbox 中是否有对象，最后的 Detector 部分用于对类进行分类，例如汽车、人、电话等