[英]WinRT Extract frames from video stream
我正在使用內置攝像頭解碼條形碼,我使用capElement.Source.CapturePhotoToStreamAsync
來捕獲預覽中的照片。 它有效,但凍結了應用程序一小段時間,感覺非常笨拙和馬車。
因此,我希望在后台進行此操作,同時至少在處理照片時留下響應式用戶界面。
到目前為止,我想出了這個來捕獲視頻流:
private async void ScanInBackground()
{
bool failedScan = true;
var stream = new InMemoryRandomAccessStream();
await capElement.Source.StartRecordToStreamAsync(MediaEncodingProfile.CreateWmv(VideoEncodingQuality.HD1080p), stream);
while(failedScan)
{
Byte[] bytes = await GetBytesFromStream(stream);
//How to split the bytes into frames?
Task.Delay(50);
}
Dispatcher.RunAsync(CoreDispatcherPriority.Low,() => StopCap());
}
以及從流中獲取字節的方法:
public static async Task<byte[]> GetBytesFromStream(IRandomAccessStream randomStream)
{
var reader = new DataReader(randomStream.GetInputStreamAt(0));
var bytes = new byte[randomStream.Size];
try
{
await reader.LoadAsync((uint)randomStream.Size); reader.ReadBytes(bytes);
}
catch(Exception ex)
{
Logger.LogExceptionAsync(ex, "GetBytesFromStream");
}
return bytes;
}
從ScanInBackground
的評論中,您可以看到我不知道如何將流分割為照片/幀。
Microsoft github頁面上有一個相關的示例,盡管它們的目標是Windows 10.您可能有興趣遷移項目以獲得此功能。
GetPreviewFrame :此示例將捕獲預覽幀而不是完整的照片。 一旦它有一個預覽框架,它就可以讀取和編輯它上面的像素。
以下是相關部分:
private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
// Get information about the preview
var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
// Create the video frame to request a SoftwareBitmap preview frame
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);
// Capture the preview frame
using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame
SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;
// Add a simple green filter effect to the SoftwareBitmap
EditPixels(previewFrame);
}
}
private unsafe void EditPixels(SoftwareBitmap bitmap)
{
// Effect is hard-coded to operate on BGRA8 format only
if (bitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8)
{
// In BGRA8 format, each pixel is defined by 4 bytes
const int BYTES_PER_PIXEL = 4;
using (var buffer = bitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
using (var reference = buffer.CreateReference())
{
// Get a pointer to the pixel buffer
byte* data;
uint capacity;
((IMemoryBufferByteAccess)reference).GetBuffer(out data, out capacity);
// Get information about the BitmapBuffer
var desc = buffer.GetPlaneDescription(0);
// Iterate over all pixels
for (uint row = 0; row < desc.Height; row++)
{
for (uint col = 0; col < desc.Width; col++)
{
// Index of the current pixel in the buffer (defined by the next 4 bytes, BGRA8)
var currPixel = desc.StartIndex + desc.Stride * row + BYTES_PER_PIXEL * col;
// Read the current pixel information into b,g,r channels (leave out alpha channel)
var b = data[currPixel + 0]; // Blue
var g = data[currPixel + 1]; // Green
var r = data[currPixel + 2]; // Red
// Boost the green channel, leave the other two untouched
data[currPixel + 0] = b;
data[currPixel + 1] = (byte)Math.Min(g + 80, 255);
data[currPixel + 2] = r;
}
}
}
}
}
並在課外宣布:
[ComImport]
[Guid("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
void GetBuffer(out byte* buffer, out uint capacity);
}
當然,您的項目必須允許所有這些的不安全代碼才能工作。
仔細查看示例,了解如何獲取所有詳細信息。 或者,要進行演練,您可以觀看最近//構建/會議中的攝像機會話 ,其中包括一些相機示例的演練。
我相信顯示媒體預覽和處理不同的可能異常是必要的,這里有一個簡單的例子,如何做到這一點,
假設您有以下UI,其中CaptureElement
用於顯示預覽,而Image
控件用於顯示捕獲的圖片,
mc:Ignorable="d" Loaded="MainPage_OnLoaded">
<Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}" VerticalAlignment="Stretch" HorizontalAlignment="Stretch">
<Grid.ColumnDefinitions>
<ColumnDefinition Width="auto"/>
<ColumnDefinition Width="auto"/>
</Grid.ColumnDefinitions>
<Grid.RowDefinitions>
<RowDefinition Height="auto"/>
<RowDefinition Height="auto"/>
</Grid.RowDefinitions>
<CaptureElement x:Name="PreviewElement" Width="400" Height="400" Grid.Column="0" Grid.Row="0"/>
<Image x:Name="ImageElement" Width="400" Height="400" Grid.Column="1" Grid.Row="0"/>
<Button Click="TakePhoto_Click" Content="Take Photo" Grid.Column="0" Grid.Row="1" Grid.ColumnSpan="2" HorizontalAlignment="Stretch" />
</Grid>
在后面的代碼聲明一個mediaCapture字段,
private MediaCapture _mediaCapture;
然后在頁面加載的事件處理程序中你需要
並啟動凸輪預覽
private async void MainPage_OnLoaded(object sender, RoutedEventArgs e) { //Start the device try { _mediaCapture = new MediaCapture(); _mediaCapture.RecordLimitationExceeded += MediaCapture_RecordLimitationExceeded; _mediaCapture.Failed += MediaCapture_Failed; await _mediaCapture.InitializeAsync(); } catch (UnauthorizedAccessException ex) { (new MessageDialog("Set the permission to use the webcam")).ShowAsync(); } catch (Exception ex) { (new MessageDialog("Can't initialize the webcam !")).ShowAsync(); } //Start the preview if (_mediaCapture != null) { try { PreviewElement.Source = _mediaCapture; await _mediaCapture.StartPreviewAsync(); } catch (Exception ex) { (new MessageDialog("Something went wrong !")).ShowAsync(); } } } private async void MediaCapture_Failed(MediaCapture sender, MediaCaptureFailedEventArgs errorEventArgs) { await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => (new MessageDialog("Media capture failed")).ShowAsync()); } private async void MediaCapture_RecordLimitationExceeded(MediaCapture sender) { await _mediaCapture.StopRecordAsync(); await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => (new MessageDialog("Record limitation exceeded")).ShowAsync()); }
最后在這里如何正確地拍攝,每一件事都是異步的,所以沒有任何延遲或者無論如何
private async void TakePhoto_Click(object sender, RoutedEventArgs e)
{
if (_mediaCapture != null)
{
try
{
ImageEncodingProperties encodingProperties = ImageEncodingProperties.CreateJpeg();
WriteableBitmap bitmap = new WriteableBitmap((int)ImageElement.Width, (int)ImageElement.Height);
using (var imageStream = new InMemoryRandomAccessStream())
{
await this._mediaCapture.CapturePhotoToStreamAsync(encodingProperties, imageStream);
await imageStream.FlushAsync();
imageStream.Seek(0);
bitmap.SetSource(imageStream);
await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal,
() =>
{
ImageElement.Source = bitmap;
});
}
}
catch (Exception ex)
{
(new MessageDialog("Something went wrong !")).ShowAsync();
}
}
}
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.