[英]Too High CPU Footprint of OpenCV Text Overlay on FHD Video Stream
I want to display a FHD live-stream (25 fps) and overlay some (changing) text.我想显示 FHD 实时流 (25 fps) 并覆盖一些(变化的)文本。 For this I essentially use the code below.
为此,我基本上使用下面的代码。
Basically it is基本上是
cv::putText
skipped here) cv::putText
)delay
delay
的倍数,则显示帧but the code is super super slow compared to eg mpv
and consumes way to much cpu-time ( cv::useOptimized() == true
).但是与例如
mpv
相比,该代码超级慢,并且消耗大量 cpu 时间( cv::useOptimized() == true
)。
So far delay
is my inconvenient fiddle-parameter to somehow make it feasible.到目前为止,
delay
是我不方便的小提琴参数,以某种方式使其可行。
delay == 1
results in 180 % CPU usage (full frame-rate) delay == 1
导致180 % CPU 使用率(全帧率)delay == 5
results in 80 % CPU usage delay == 5
导致 80 % CPU 使用率But delay == 5
or 5 fps is really sluggish and actually still too much cpu load.但是
delay == 5
或 5 fps 真的很慢,实际上仍然有太多的 cpu 负载。
How can I make this code faster or otherwise better or otherwise solve the task (I'm not bound to opencv)?
我怎样才能使这个代码更快或更好或以其他方式解决任务(我不绑定到opencv)?
Ps Without cv::imshow
the CPU usage is less than 30 %, regardless of delay
. Ps没有
cv::imshow
的 CPU 使用率小于 30%,不管delay
。
#include <opencv2/opencv.hpp>
#include <X11/Xlib.h>
// process ever delayth frame
#define delay 5
Display* disp = XOpenDisplay(NULL);
Screen* scrn = DefaultScreenOfDisplay(disp);
int screen_height = scrn->height;
int screen_width = scrn->width;
int main(int argc, char** argv){
cv::VideoCapture cap("rtsp://url");
cv::Mat frame;
if (cap.isOpened())
cap.read(frame);
cv::namedWindow( "PREVIEW", cv::WINDOW_NORMAL );
cv::resizeWindow( "PREVIEW", screen_width, screen_height );
int framecounter = 0;
while (true){
if (cap.isOpened()){
cap.read(frame);
framecounter += 1;
// Display only delay'th frame
if (framecounter % delay == 0){
/*
* cv::putText
*/
framecounter = 0;
cv::imshow("PREVIEW", frame);
}
}
cv::waitKey(1);
}
}
I now found out about valgrind
(repository) and gprof2dot
( pip3 install --user gprof2dot
):我现在发现了
valgrind
(repository) 和gprof2dot
( pip3 install --user gprof2dot
):
valgrind --tool=callgrind /path/to/my/binary # Produced file callgrind.out.157532
gprof2dot --format=callgrind --output=out.dot callgrind.out.157532
dot -Tpdf out.dot -o graph.pdf
That produced a wonderful graph saying that over 60 % evaporates on cvResize
.这产生了一个精彩的图表,表明
cvResize
超过 60% 蒸发了。 And indeed, when I comment out cv::resizeWindow
, the cpu usage lowers from 180 % to ~ 60 %.事实上,当我注释掉
cv::resizeWindow
时,cpu 使用率从 180% 降低到 ~ 60%。
Since the screen has a resolution of 1920 x 1200 and the stream 1920 x 1080, it essentially did nothing but burning CPU cycles.由于屏幕的分辨率为 1920 x 1200,而 stream 为 1920 x 1080,它基本上只消耗 CPU 周期。
So far, this is still fragile.到目前为止,这仍然是脆弱的。 As soon as I switch it to full-screen mode and back, the cpu load goes back to 180 %.
一旦我将它切换到全屏模式并返回,cpu 负载就会回到 180%。
To fix this, it turned out that I can either disable resizing completely with cv::WINDOW_AUTOSIZE
...为了解决这个问题,事实证明我可以使用
cv::WINDOW_AUTOSIZE
完全禁用调整大小 ...
cv::namedWindow( "PREVIEW", cv::WINDOW_AUTOSIZE );
... or -- as Micka suggested -- on OpenCV versions compiled with OpenGL support ( -DWITH_OPENGL=ON
, my Debian repository version was not), use... ...或 - 正如Micka建议的那样 - 在 OpenCV 版本上编译了 OpenGL 支持(
-DWITH_OPENGL=ON
,我的 Debian 版本是...
cv::namedWindow( "PREVIEW", cv::WINDOW_OPENGL );
... to offload the rendering to the GPU, what turns out to be even faster together with resizing (55 % CPU compared to 65 % for me). ...将渲染卸载到 GPU,结果证明与调整大小一起更快(55% CPU,而我为 65%)。 It just does not seem to work together with
cv::WINDOW_KEEPRATIO
.*它似乎无法与
cv::WINDOW_KEEPRATIO
一起使用。*
Furthermore, it turns out that cv:UMat
can be used as a drop-in replacement for cv:Mat
which additionally boosts the performance (as seen by ps -e -o pcpu,args
):此外,事实证明
cv:UMat
可以用作cv:Mat
的替代品,这还可以提高性能(如ps -e -o pcpu,args
):
[*] So we have to manually scale it and take care of the aspect ratio. [*] 所以我们必须手动缩放它并注意纵横比。
float screen_aspratio = (float) screen_width / screen_height;
float image_aspratio = (float) image_width / image_height;
if ( image_aspratio >= screen_aspratio ) { // width limited, center window vertically
cv::resizeWindow("PREVIEW", screen_width, screen_width / image_aspratio );
cv::moveWindow( "PREVIEW", 0, (screen_height - image_height) / 2 );
}
else { // height limited, center window horizontally
cv::resizeWindow("PREVIEW", screen_height * image_aspratio, screen_height );
cv::moveWindow( "PREVIEW", (screen_width - image_width) / 2, 0 );
}
One thing that pops is you're creating a new window and resizing it every time you want to display something.突然出现的一件事是您正在创建一个新的 window 并在每次要显示某些内容时调整它的大小。
move these lines移动这些线
cv::namedWindow( "PREVIEW", cv::WINDOW_NORMAL );
cv::resizeWindow( "PREVIEW", screen_width, screen_height );
to before your while(true)
and see it that solves this在你的
while(true)
之前,看看它解决了这个问题
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.