[英]How to minimize the lag from the live stream using OpenCV/EmguCV WPF
I am using EmguCV to get a live stream from a high-resolution webcam.我正在使用 EmguCV 从高分辨率网络摄像头获取实时流。 (full HD) The issue is that there is a significant lag in the video.
(全高清)问题在于视频中存在明显滞后。 I compared with the Windows camera app and my application is much more delayed.
我与 Windows 相机应用程序相比,我的应用程序延迟了很多。
Here is the code snippet I am using to get the live stream and show on the canvas.这是我用来获取实时流并在画布上显示的代码片段。 I am wondering if I am missing anything important to minimize the lag.
我想知道我是否遗漏了任何重要的东西来最小化滞后。 If anybody is experienced, please help me.
如果有人有经验,请帮助我。
void init_camera()
{
m_capture= new VideoCapture(0);
m_capture.ImageGrabbed += ProcessFrame;
}
void ProcessFrame()
{
if (m_capture_in_progress)
return;
m_capture_in_progress = true;
if (m_capture != null && m_capture.Ptr != IntPtr.Zero)
{
m_capture.Retrieve(m_frame, 0);
if (m_frame != null && m_frame.Bitmap != null)
{
if (IsRecording && m_video_writer != null && m_video_writer.IsOpened)
{
try
{
m_video_writer.Write(m_frame);
}
catch (Exception ex)
{
log(ex.Message + "\n" + ex.StackTrace);
}
}
this.Dispatcher.Invoke(() =>
{
using (var stream = new MemoryStream())
{
m_frame.Bitmap.Save(stream, System.Drawing.Imaging.ImageFormat.Jpeg);
using (var outstream = new MemoryStream(stream.ToArray()))
{
BitmapImage bitmap = new BitmapImage();
bitmap.BeginInit();
bitmap.StreamSource = new MemoryStream(stream.ToArray());
bitmap.CacheOption = BitmapCacheOption.OnLoad;
bitmap.EndInit();
ui_canvas.Background = new ImageBrush(bitmap);
}
};
});
}
}
m_capture_in_progress = false;
}
OP is referring some kind of laggs while drawing a camera capture stream from EmguCv into canvas
wpf control. OP 在从 EmguCv 将相机捕获流绘制到
canvas
wpf 控件中时指的是某种滞后。 OP also states that lagg disappears when streaming with third party software. OP 还指出,使用第三方软件流式传输时,滞后会消失。
This is caused by low performance in OP's code.这是由 OP 代码的低性能引起的。
After wataching the posted code sample I would suggest the following improvements:观看发布的代码示例后,我建议进行以下改进:
Maybe your device is supporting higher fps rates, try to increase this value.也许您的设备支持更高的 fps 速率,请尝试增加此值。
image
image
绘制位图As far as I know, drawing a bitmap over a canvas
is slower than drawing it over an image
control.据我所知,在
canvas
绘制位图比在image
控件上绘制要慢。 A good solution would be to overlap both controls so you can paint the frames over the image
and draw your shapes over the canvas
as they are well overlapped.一个很好的解决方案是重叠两个控件,这样您就可以在
image
绘制框架并在canvas
绘制您的形状,因为它们重叠得很好。
Keep simple and try to avoid so much control structures when you are inside a critical program block.当您处于关键程序块中时,请保持简单并尽量避免使用过多的控制结构。 Specially in program blocks that are executed when an external devices fires an event.
特别是在外部设备触发事件时执行的程序块中。
As far as I can see, you are locking the ProcessFrame
event with a bool variable called m_capture_in_progress
which is set to true while you are drawing the frame and it is freed after you finish the drawing.据我所知,您正在使用名为
m_capture_in_progress
的 bool 变量锁定ProcessFrame
事件,该变量在您绘制框架时设置为 true,并在您完成绘图后释放。 This can cause that next incoming frames are not painted because the method is still blocked by the previous frame.这可能会导致下一个传入帧未绘制,因为该方法仍被前一帧阻止。 This can cause a low performance issue as many frames get lost and not painted.
这可能会导致性能低下的问题,因为许多帧丢失且未绘制。
Use the Image control but set its Source Property to a WriteableBitmap instead of a BitmapImage.使用 Image 控件,但将其 Source 属性设置为WriteableBitmap而不是 BitmapImage。 Then lock the bitmap, copy the pixel data from your EmguCV Mat to the backbuffer of the WriteableBitmap, call AddDirtyRect method and finally unlock the Bitmap.
然后锁定位图,将 EmguCV Mat 中的像素数据复制到 WriteableBitmap 的后台缓冲区,调用 AddDirtyRect 方法,最后解锁位图。 The final unlock call in combination with the AddDirtyRect will trigger a redraw of the UI image.
最后的解锁调用与 AddDirtyRect 结合将触发 UI 图像的重绘。
Adavantages:优点:
Your copy, encode and decode data to often:您经常复制、编码和解码数据:
// Copies and encodes pixel data to jpeg stream
m_frame.Bitmap.Save(stream, System.Drawing.Imaging.ImageFormat.Jpeg);
// Copies jpeg stream
using (var outstream = new MemoryStream(stream.ToArray()))
{
BitmapImage bitmap = new BitmapImage();
bitmap.BeginInit();
//Copies jpeg stream again
bitmap.StreamSource = new MemoryStream(stream.ToArray());
bitmap.CacheOption = BitmapCacheOption.OnLoad;
// Triggers jpeg stream decoding
bitmap.EndInit();
ui_canvas.Background = new ImageBrush(bitmap);
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.