在流式传输时如何处理Azure Media Service Live Stream视频？

Question

I am creating an application which takes video from a camera hosted on the web, runs it through a computer vision algorithm to detect humans (written in C# using EmguCV's OpenCV wrapper) and streams the processed video to an ASP.NET client.

The process I believed would work was to have Azure Media Services create a live stream channel for the video, and somewhere in the process inject my code to process the video. The algorithm uses a SQL database for much of its decision making, and so I thought to put it in a WebJob and have it process video as it is put in storage. I would much rather process it somewhere in the Azure Media Services process, instead of using a WebJob.

My question is: is there a way to process the video as it is coming in, so what is seen in storage is the processed video with boxes around the people (boxes placed by my algorithm which takes a frame as input and outputs a frame)? If so, where can I put my logic to do this, in the encoder setup?

Also, if you have another way of doing it please let me know! I am open to ideas! I plan on scaling this app to use more than one camera as input, and the client should be able to switch between feeds. This is off topic from my question but is a consideration. I know it is possible to have a WebJob take the video out of storage, process it, and put it back, but the app loses the "Live" aspect then.

Technology Stack: Azure SQL DB created Azure Website created Azure Media Services and Storage created Possible Azure WebJob to handle algorithm?

Thank you so much in advance for any help!

Answer 1

As of now Azure Media Services is not allowing to plug in user defined code into processing pipeline. You can select existing processor or utilize 3d party encoders which are currently presented through azure marketplace.

For now(based on requirements you have) i think you need to have a proxy VM which doing face recognition of incoming stream and redirect processed stream to Azure Media Services live channel. NGIX web server + ffmpeg + OpenCV can be a good solution to look into.