[英]Efficiently calculating distance of pixels to their corresponding epipolar lines
I'm trying to calculate the distance of two corresponding pixels in a stereo camera setup to their respective epipolar lines.我正在尝试计算立体相机设置中两个相应像素到它们各自的极线的距离。 The code I Implemented for this purpose looks is the following:
我为此目的实现的代码如下所示:
#include <iostream>
#include <vector>
#include <opencv2/calib3d/calib3d.hpp>
float calculateDistanceToEpiLineSum(const cv::Mat2f& left_candidate, const cv::Mat2f& right_candidate, const cv::Matx33f& fundamental_mat) {
// Calculate epipolar lines
cv::Mat epiLineRight=cv::Mat(1,3,CV_32FC1);
cv::Mat epiLineLeft=cv::Mat(1,3,CV_32FC1);
cv::computeCorrespondEpilines(left_candidate,2,fundamental_mat,epiLineRight);
cv::computeCorrespondEpilines(right_candidate,1,fundamental_mat,epiLineLeft);
// Calculate distances of the image points to their corresponding epipolar line
float distance_left_im=std::abs(epiLineLeft.at<float>(0)*left_candidate[0][0][0]+
epiLineLeft.at<float>(1)*left_candidate[0][0][1]+
epiLineLeft.at<float>(2))/
std::sqrt(std::pow(epiLineLeft.at<float>(0),2.f)+std::pow(epiLineLeft.at<float>(1),2.f));
float distance_right_im=std::abs(epiLineRight.at<float>(0)*right_candidate[0][0][0]+
epiLineRight.at<float>(1)*right_candidate[0][0][1]+
epiLineRight.at<float>(2))/
std::sqrt(std::pow(epiLineRight.at<float>(0),2.f)+std::pow(epiLineRight.at<float>(1),2.f));
return distance_left_im+distance_right_im;
}
int main()
{
cv::Matx33f fundamental_mat=cv::Matx33f{-0.000000234008931f,-0.000013193232976f, 0.010025275471910f,
-0.000017896532640f, 0.000009948056751f, 0.414125924093639f,
0.006296743991557f,-0.411007947095269f,-4.695511356888332f};
cv::Vec2f left_candidate_vec(135.,289.);
cv::Vec2f right_candidate_vec(205.,311.);
cv::Mat2f left_candidate(left_candidate_vec);
cv::Mat2f right_candidate(right_candidate_vec);
float distance_sum=calculateDistanceToEpiLineSum(left_candidate,right_candidate,fundamental_mat);
std::cout<<"The sum of the distances equals "<<distance_sum<<" pixels\n";
return 0;
}
The problem that I'm facing is that I will have to perform this operation possibly thousands of times each second.我面临的问题是我将不得不每秒执行数千次此操作。 I am aware that
cv::computeCorrespondEpilines
s first input can be a vector of pixels which allows a more vectorized approach and would probably speed things up a little.我知道
cv::computeCorrespondEpilines
的第一个输入可以是一个像素向量,它允许采用更矢量化的方法,并且可能会加快速度。 The problem is, that I can not make use of this, because I'm not working with conventional cameras, but event-based sensors and therefore I will receive the pixels asynchronously (instead of receiving frames).问题是,我无法使用它,因为我不是使用传统相机,而是使用基于事件的传感器,因此我将异步接收像素(而不是接收帧)。
Now I would like to know the following:现在我想知道以下内容:
calculateDistanceToEpiLineSum
that influence the performance of the function in a bad way? calculateDistanceToEpiLineSum
中是否存在严重影响 function 性能的重大缺陷? Is it maybe a good idea, to not use OpenCV functions here, but just implement computeCorrespondEpilines
myself?computeCorrespondEpilines
可能是个好主意吗?ax+by+c=0
, but assuming a resolution of 480x360 of each camera, that would give me two 480x360x3 matrices which is quite big.ax+by+c=0
将线参数存储在三元组中,但假设每个相机的分辨率为 480x360,这将给我两个 480x360x3 矩阵相当大。 Is it viable to do it like this anyway, or is there any better approach?Questions of time performance are impossible to answer without knowing what your computational constraints are, so what follows is a very rough estimate.如果不知道您的计算限制是什么,就无法回答时间性能问题,因此以下是一个非常粗略的估计。
At bottom, computing a distance of one point of a candidate matched pair of pixels from its associated epipolar line cost roughly:在底部,计算候选匹配像素对的一个点与其关联的极线的距离大致成本:
So about 70 flops for the pair.所以这对大约有 70 次翻牌。
What this all means in seconds depends, at a minimum, on your processor clock cycle.这一切在几秒钟内意味着什么,至少取决于您的处理器时钟周期。 A Skylake IA64 can do 16 DP vectorized flops/cycle, so call it 5 cycles for the pair.
Skylake IA64 可以执行 16 DP 矢量化触发器/周期,因此将其称为 5 个周期。 At 3 GHz that takes less than 2 ns.
在 3 GHz 时,所需时间不到 2 ns。 Taking a generous margin on that, let's call it 10 ns total.
在这方面留有很大的余地,我们称之为总共 10 ns。
You say you will have to perform this computation "thousand of times" per second.你说你必须每秒执行“数千次”这个计算。 At 10ns per pair, you can do 100 million of them per second.
在每对 10ns 的情况下,您每秒可以执行1 亿次。
Given the above, are you sure this particular operation is going to be the bottleneck and not, for example, the I/O from the camera instead (including image decoding)?鉴于上述情况,您确定此特定操作将成为瓶颈,而不是例如来自相机的 I/O(包括图像解码)?
Word of advice: learn to use a good microbenchmarking framework to get actual performance numbers for your hardware.忠告:学习使用良好的微基准测试框架来获取硬件的实际性能数据。 I recommend good old Google Benchmark .
我推荐旧的Google Benchmark 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.