Javascript MediaSource API和H264视频

Question

I have a problem playing a H264 video using javascript MediaSource Extension API. 我在使用javascript MediaSource Extension API播放H264视频时遇到问题。

I'll describe the scenario with details below. 我将在下面详细介绍该方案。

I've already successfully achieved the result playing audio and video source of vp8, vp9, opus and vorbis codec, also from a range request ( if server has the capability, using any byte range ) or chunked files, chunks done using shaka packager. 我已经成功地实现了播放vp8，vp9，opus和vorbis编解码器的音频和视频源的结果，也来自范围请求（如果服务器具有使用任何字节范围的功能）或chunked文件，使用shaka packager完成的块。

The problem comes when the source is an H264 video, in details in my case codecs are avc1.64001e and mp4a.40.2, full codec string is video/mp4;codecs="avc1.64001e, mp4a.40.2" but the issue still happens with any other avc1 codec. 当源是H264视频时出现问题，在我的情况下，编解码器是avc1.64001e和mp4a.40.2，完整的编解码器字符串是视频/ mp4;编解码器=“avc1.64001e，mp4a.40.2”，但问题仍然存在与任何其他avc1编解码器。

What I am trying to do is to play a 10 megabytes chunk of the full video, chunk generated by a byterange curl request saving the response locally using -o. 我想要做的是播放完整视频的10兆字节块，由字节范围卷曲请求生成的块使用-o在本地保存响应。

Below the stream info from shaka packager passing this file as input 在shaka packager的流信息下面传递此文件作为输入

[0530/161459:INFO:demuxer.cc(88)] Demuxer::Run() on file '10mega.mp4'.
[0530/161459:INFO:demuxer.cc(160)] Initialize Demuxer for file '10mega.mp4'.

File "10mega.mp4":
Found 2 stream(s).
Stream [0] type: Video
 codec_string: avc1.64001e
 time_scale: 17595
 duration: 57805440 (3285.3 seconds)
 is_encrypted: false
 codec: H264
 width: 720
 height: 384
 pixel_aspect_ratio: 1:1
 trick_play_factor: 0
 nalu_length_size: 4

Stream [1] type: Audio
 codec_string: mp4a.40.2
 time_scale: 44100
 duration: 144883809 (3285.3 seconds)
 is_encrypted: false
 codec: AAC
 sample_bits: 16
 num_channels: 2
 sampling_frequency: 44100
 language: und

Packaging completed successfully.

The chunk is playable with external media player applications ( like VLC ) and more important, it plays without problem adding it to the webpage using the < source > tag. 该块可以与外部媒体播放器应用程序（如VLC）一起播放，更重要的是， 使用<source>标签将其添加到网页时可以正常播放。

This is the error I can see in the Chrome console 这是我在Chrome控制台中看到的错误

Uncaught (in promise) DOMException: Failed to load because no supported source was found.

Here below the html and js code if you want to reproduce ( I did all local tests using the built-in php7.2 dev server ) 下面是html和js代码，如果你想重现（我使用内置的php7.2开发服务器进行了所有本地测试）

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8"/>
    <title>VideoTest</title>
    <link rel="icon" href="/favicon.ico" />
    <script type="text/javascript" src="/script.js"></script>

    <style>
        video {
            width: 98%;
            height: 300px;
            border: 0px solid #000;
            display: flex;
        }
    </style>
</head>
<body>

<div id="videoContainer">
    <video controls></video>
</div>

<video controls>
    <source src="/media/10mega.mp4" type="video/mp4">
</video>

</body>
</html>

And here below the JS code ( scripjs ) 以下是JS代码（scripjs）

class MediaTest {
    constructor() {

    }

    init(link) {
        this.link = link;
        this.media = new MediaSource();
        this.container = document.getElementsByTagName('video')[0];
        this.container.src = window.URL.createObjectURL(this.media);

        return new Promise(resolve => {
            this.media.addEventListener('sourceopen', (e) => {
                this.media = e.target;
                return resolve(this);
            });
        });
    }

    addSourceBuffer() {
        let codec = 'video/mp4;codecs="avc1.64001e, mp4a.40.2"';
        let sourceBuffer = this.media.addSourceBuffer(codec);

        // These are the same headers sent by the < source > tag
        // with or without the issue remains
        let headers = new Headers({
            'Range': 'bytes=0-131072',
            'Accept-Encoding': 'identity;q=1, *;q=0'
        });

        let requestData = {
            headers: headers
        };
        let request = new Request(this.link, requestData);

        return new Promise(resolve => {
            fetch(request).then((response) => {
                if(200 !== response.status) {
                    throw new Error('addSourceBuffer error with status ' + response.status);
                }

                return response.arrayBuffer();
            }).then((buffer) => {
                sourceBuffer.appendBuffer(buffer);
                console.log('Buffer appended');
                return resolve(this);
            }).catch(function(e) {
                console.log('addSourceBuffer error');
                console.log(e);
            });
        });
    }

    play() {
        this.container.play();
    }
}

window.addEventListener('load', () => {
    let media = new MediaTest();
    media.init('/media/10mega.mp4').then(() => {
        console.log('init ok');
        return media.addSourceBuffer();
    }).then((obj) => {
        console.log('play');
        media.play();
    });
});

What I want to achieve is to play the file with MediaSource API since it plays well using < source > tag. 我想要实现的是使用MediaSource API播放文件，因为它使用<source>标签很好地播放。 I don't want to demux and re-encode it, but use it as is. 我不想对它进行解复用和重新编码，而是按原样使用它。

Here below the error dump taken from chrome://media-internals 下面是从chrome：// media-internals中获取的错误转储

render_id: 180 player_id: 11 pipeline_state: kStopped event: WEBMEDIAPLAYER_DESTROYED render_id：180 player_id：11 pipeline_state：kStopped事件：WEBMEDIAPLAYER_DESTROYED

To reproduce I think it is possible to use any H264 video that has audio and video track within it. 为了重现，我认为可以使用任何具有音频和视频轨道的H264视频。

This question is strictly related with this other question I've found H264 video works using src attribute. 这个问题与我发现H264视频使用src属性的其他问题严格相关。 Same video fails using the MediaSource API (Chromium) but it is from 4 years ago so I decided not to answer there. 使用MediaSource API（Chromium）的同一视频失败，但它是从4年前开始的，所以我决定不回答那里。

Does anybody have some idea about this issue? 有没有人对这个问题有所了解？ Is there any way to solve it or h264 It is just not compatible with MSE? 有没有办法解决它或h264它与MSE不兼容？

Thanks in advance 提前致谢

Answer 1

Its not the codec, its the container. 它不是编解码器，它是容器。 MSE requires fragmented mp4 files. MSE需要碎片化的mp4文件。 Standard mp4 is not supported. 不支持标准mp4。 for standard mp4 you must use <video src="my.mp4"> 对于标准mp4，您必须使用<video src="my.mp4">

Javascript MediaSource API和H264视频

问题描述

1 个解决方案

解决方案1
2 2018-05-31 01:36:54

Javascript MediaSource API和H264视频

问题描述

1 个解决方案

解决方案1 2 2018-05-31 01:36:54

解决方案1
2 2018-05-31 01:36:54