简体繁体 English

Opencv光流跟踪：停止条件

[英]Opencv Optical flow tracking: stop condition

原文 2015-08-26 14:40:41 2 1 c++/ opencv/ face-detection/ motion/ opticalflow

I'm currently trying to implement a face tracking by using optical flow with opencv. 我目前正在尝试通过将光流与opencv结合使用来实现人脸跟踪。

To achieve this, I detect faces with the openCV face detector, I determine features to track on the detected areas by calling goodFeaturesToTrack and I operate tracking by calling calcOpticalFlowPyrLK . 为了实现这一点，我检测脸部与OpenCV的面部检测器，我确定功能，以跟踪所检测到的区域，通过调用goodFeaturesToTrack和我操作通过调用跟踪calcOpticalFlowPyrLK 。

It gives good results. 它给出了很好的结果。

However, I'd like to know when the face I'm currently tracking is not visible anymore (the person leaves the room, is hidden behind an object or another person, ...) but calcOpticalFlowPyrLK tells me nothing about it. 但是，我想知道什么时候我当前正在跟踪的脸不再可见（那个人离开房间，隐藏在一个物体或另一个人的后面，...），但是calcOpticalFlowPyrLK告诉了我什么。

The status parameter of the calcOpticalFlowPyrLK function rarely reports errors concerning a tracked feature (so, if the person disappear, I will still have a good amount of valid features to track). calcOpticalFlowPyrLK函数的status参数很少报告与跟踪特征有关的错误（因此，如果此人消失了，我仍然要跟踪大量有效特征）。

I've tried to calculate the directional vectors for each feature to determine the move between the previous and the actual frame for each feature of the face (for example, determining that some point of the face has move to the left between the two frames) and to calculate the variance of these vectors (if vectors are mostly different, variance is high, otherwise it is not) but it did not give the expected results (good in some situation, but bad in other cases). 我尝试计算每个特征的方向矢量，以确定该脸部每个特征在上一帧与实际帧之间的移动（例如，确定脸部某个点已在两个帧之间向左移动）并计算这些向量的方差（如果向量大部分不同，则方差高，否则就没有），但是没有给出预期的结果（在某些情况下是好的，在其他情况下是不好的）。

What could be a good condition to determine whether the optical flow tracking has to be stopped or not? 确定是否必须停止光流跟踪的好条件是什么？

I've thought of some possible solutions like these ones: 我想到了一些可能的解决方案，例如：

variance of the distance for the vectors of each tracked feature (if the move is linear, distances should be nearly the same, but if something happened, distances will be different). 每个被跟踪要素的矢量的距离方差（如果移动是线性的，则距离应几乎相同，但是如果发生某些情况，则距离将不同）。
Comparing the shape and size of the area containing the original position of the tracked features with the area containing the current one. 将包含跟踪要素原始位置的区域的形状和大小与包含当前要素的区域进行比较。 At the beginning we have a square containing the features of the face. 开始时，我们有一个包含脸部特征的正方形。 But if the person leaves the room, it can lead to a deformation of the shape. 但是如果人离开房间，可能会导致形状变形。

1 个解决方案

You can try a bidirectional confidenze measure of your track points. 您可以尝试对跟踪点进行双向约束度量。 Therefore estimate the feature positions from img0 to img1 and than the tracked positions backwards from img1 to img0. 因此，估计从img0到img1的特征位置，然后估计从img1到img0的特征位置。 If the double tracked features near the original ( distance should be less than 1 or 0.5 pixel) than they are successfully tracked. 如果双重跟踪的要素接近原始要素（距离应小于1或0.5像素），则不能成功跟踪它们。 This is a little bit more relyable than the SSD which is used by the status flag of opencv's plk. 这比opencv plk的状态标志所使用的SSD更可靠。 If a certain amount of features could not been tracked the event raises. 如果无法跟踪一定数量的功能，则会引发该事件。