How to stream Base64 data into a video

Question

I am trying to get a video to load from a stream. I would like to be able to "play" the video even if the entire video has not "downloaded" yet. The base64 data is coming in as a stream of buffer chunks.

const videoTag = document.getElementById('videoTag');
const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';
const mediaSource = new MediaSource();

videoTag.src = URL.createObjectURL(mediaSource);

await new Promise((resolve, reject) => {
    mediaSource.addEventListener('sourceopen', function (_) {
        resolve();
    });
});

const sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);


for await (const chunk of ipfsNode.cat(CID)) {//Chunk is coming in as an array buffer
    sourceBuffer.appendBuffer(chunk);
}

sourceBuffer.addEventListener('updateend', async function (_) {
    mediaSource.endOfStream();
    $('#videoTag')[0].load();
});

I've tried using new MediaSource() and sourceBuffer.appendBuffer(chunk); to push new downloaded chunks to the video, but not only does the video not ever play, the method I am using requires that the entire video downloads before it can play.

How can I get my chunk s of base64 data into a video?

EDIT (in response to comments): Regardless of whether or not I use a String or binary steam, I am a little confused on how I get a stream into a video.

Answer 1

You have two problems: 1. how to add arbitrary data to a video stream? 2. how to stream?

1. Given that you use a mp4 container ( const mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"'; ) you can add arbitrary number of private streams , hence your encrypted stream (that can be base64 or bytes[] as well).

   From the MP4 registration authority the codecs encm (Encrypted/Protected metadata), encs (Encrypted Systems stream), enct (Encrypted Text) seems to be good candidates for your encrypted private stream .

2. ffmpeg have the ability to mux multiple streams into the final stream ; this might be the solution on the backend side. Here there is no need to have the input/private streams finished in order to be able to mux and broadcast into the final stream; the reading should also happen without reaching the end of the stream.

Answer 2

使用data: URL 并将其直接插入到视频 src 属性中。