[英]How to use MediaStreamSource to play h264 frames coming from a matroska file?
I'm trying to render frames coming from an mkv h264 file in silverlight 3 by using the MediaStreamSource. 我正在尝试使用MediaStreamSource在Silverlight 3中渲染来自mkv h264文件的帧。 Parsing the mkv file is fine, but I'm struggling with the expected values for CodecPrivateData in SL, which has to be a string, while the PrivateData info from mkv is a binary element. 解析mkv文件很好,但是我正在为SL中的CodecPrivateData的期望值挣扎,它必须是字符串,而来自mkv的PrivateData信息是一个二进制元素。 Also, I'm not sure about in which form the frames should be given to SL (ie, the way they are stored in mkv / mp4, or transcoded as NALU) Would anyone have any info on this? 另外,我不确定应以哪种形式将帧提供给SL(即,它们以mkv / mp4格式存储或以代码转换为NALU的方式)是否有人对此有任何信息?
After similar problems of my own and much head-scratching, I am able to answer this question. 在经历了我自己的类似问题并为之苦恼后,我能够回答这个问题。
In ReportOpenMediaCompleted(), when setting up your video stream description, you can ignore the CodecPrivateData attribute string, despite what the documentation says. 在ReportOpenMediaCompleted()中,设置视频流描述时,尽管文档中有说明,您仍可以忽略CodecPrivateData属性字符串。 It's not required. 不是必需的 (assuming your stream of NAL units includes SPS and PPS units) (假设您的NAL单位流包括SPS和PPS单位)
You should send one NAL unit back to the MediaElement for each GetSampleAsync() request. 对于每个GetSampleAsync()请求,应将一个NAL单元发送回MediaElement。 This includes non-picture NAL units, eg SPS / PPS units. 这包括非图片NAL单位,例如SPS / PPS单位。
When you send your NAL units, ensure there are 3-byte start codes (0x00 0x00 0x01) at the beginning of each one. 发送NAL单元时,请确保每个单元的开头都有3字节的起始码(0x00 0x00 0x01)。 (This is similar to 'Annex B' format, but not quite the same thing) (这类似于“附件B”格式,但不完全相同)
In ReportGetSampleCompleted(), set the value of 'Offset' equal to the beginning of the NAL start code, not the actual data. 在ReportGetSampleCompleted()中,将“偏移”的值设置为等于NAL起始代码的开头,而不是实际数据的开头。 (in most cases this will be zero, assuming you use a fresh stream per NAL unit) (在大多数情况下,如果您使用每个NAL单位使用新的流,则该值为零)
I have blogged a little about the experience here and hope to blog more. 我已经在这里写了一些有关体验的博客,希望能写更多的博客。
According to the documentation the Codec private data should be set to 00000001 + sps + 00000001 + pps. 根据文档,编解码器专用数据应设置为00000001 + sps + 00000001 + pps。 However the documentation is wrong the value of CodecPrivateData seems to be completely ignored. 但是,文档错误 ,似乎完全忽略了CodecPrivateData的值。 Instead you need to pass the SPS and PPS NALS (with an annex b header of course) as the first and second result of GetSampleAsync
. 相反,您需要传递SPS和PPS NALS(当然带有附件b标头)作为GetSampleAsync
的第一和第二结果。
For regular media samples normal 4 byte annex b headers headers work just fine 对于常规媒体样本,正常的4字节附件b标头标头工作正常
The CodecPrivateData is the contents of the 'avcC' atom which is a child of the 'stsd' atom in an MP4 file. CodecPrivateData是MP4文件中'stsd'原子的子级'avcC'原子的内容。 You have to convert the binary data to a string. 您必须将二进制数据转换为字符串。 It will look something like this: "014D401FFFE10017674D401F925402802DD0800000030080000018478C195001000468EE32C8" 它看起来像这样:“ 014D401FFFE10017674D401F925402802DD0800000030080000018478C195001000468EE32C8”
You also have to replace the mkv/mp4 lengths to NALU. 您还必须将mkv / mp4长度替换为NALU。 I've written a little about this (to get Smooth Streaming to work for H.264 files). 我已经写了一些有关此内容的信息(以使平滑流能够用于H.264文件)。
Regards, 问候,
See: Smooth Streaming H264 请参阅: 流畅流式H264
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.