Extract part of video with html5 video. Fullscreen issues on iphone/ipad

Question

What i want to accomplish:

I have a project where i hope to accomplish what vine has done in their app. My project will be a normal website.

Here is a screenshot of what i generally want:

It should be possible for a user to record a video, take parts of it and upload it to my site. The audio should also be a part of the video.

Until now ive made a slider which loops the selected area. The current code\\prototype can be seen here: http://smn-vlp.azurewebsites.net/ Careful: There is sound.

Right now its done only with javascript and the video dom element.

Problem: Iphone uses fullscreen video no matter what i do with the selected part. On other devices it seems to work great.

Possible solution: I tried using canvas to play the video, but in order to actually get images to the canvas, the original video has to .play(). This triggers the fullscreen mode from safari once a again. I then thought about setting currentTime =+1 and get frames to a canvas without actually playing the video. But, can i save the drawn images in an array to generate into a video afterwards?

What do i do about sound if i generate video from canvas images? Does this work?

function CaptureAudio() {
    var audioContext = new webkitAudioContext();
    var gainNode = audioContext.createGain();
    gainNode.gain.value = 1;                   // Change Gain Value to test
    filter = audioContext.createBiquadFilter();
    filter.type = 2;                          // Change Filter type to test
    filter.frequency.value = 5040;            // Change frequency to test

    var source = audioContext.createMediaElementSource(video);
    source.connect(gainNode);
    gainNode.connect(filter);
    filter.connect(audioContext.destination);
    console.log(source);
}

If so, im thinking i have to keep track of the selected part of the video, and get the audio for that part, before generating the video. Can the video be generated from images, and audio together?

Now, before trying all this i would love to hear from anyone who has done something similar, so i dont follow a crazy path that cant be completed. This project has some budget limits at the moment.

Summary of questions:

1. Should i use canvas to generate a selected part of the video?
2. Can i add audio to the generated video, from the original video?
3. Is this the way to go?
4. Is it actually possible to generate the video on iphone without going fullscren?
5. I would love other general suggestions on how to accomplish this.

Answer 1

According to this post , you don't have any solution for your application to work directly "as a website". Users must save it to their dashboard, or you have to make an iOS app with a webview...

Cheers.

Answer 2

You have a lot of questions that should be separate, not just here, but I'll answer a couple:

Playing the video inline on the iPhone

The iPhone and iPod force fullscreen video playback in Safari (and in apps that make use of its unmodified WebView) but you can work around this issue by simulating the playback by skimming the video instead of actually .play() 'ing it.

I wrote a module that takes care of playing the video inline and synchronizing it to the audio (but it also works on videos without a sound track): iphone-inline-video

With the suggested module, you can:

// videoElement is the <video> element
makeVideoPlayableInline(videoElement);

// iOS needs user interaction to load/play a video,
// so you'll need a start button or similar
// You only need to load the video, so .play and then .pause it
startButton.ontouchstart = function () {
    videoElement.play();
    setTimeout(function () { videoElement.pause() }, 10);
}

Extracting frames

Once the video is loaded, you can just do

// skim to 4.3 seconds
video.currentTime = 4.3;

// extract the frame to your canvas
canvas.drawImage(video, 0, 0, video.videoWidth, video.videoHeight);

Now your canvas will have the extracted frame. Save the image from the canvas with something like array.push(ctx.getImageData()) (careful, this is really memory-intensive, if you save many frames you can crash the browser)