Android：如何使用Mediacodec将两个重叠的视频保存为一个？

Question

I want to do something similar to this on android.

I want to load video from file and display it twice with some transformations (mostly resize). And i want to encode it as one video file.

Is it possible to do it in native android? With mediacodec or in any other way?

I know there is a ffmpeg but i have trouble compiling it and working with Xamarin.

Answer 1

MMSF will allow you to display one video twice on the screen. MediaCodec is used for all video decode operations. No special decode/extra libraries are needed. MMSF only uses existing Android Java APIs (MediaCodec/GLES). The goal of displaying a single video to multiple geoemtry can be accomplished within the fragment shader with relative ease. MMSF Sample #2 illustrates how to display multiple videos, but could be tweaked to make a single video display 1 or more times to the screen. Within the fragment FragmentShader.java you would want to change the texture samplers to all sample from the same source instead of unique samplers. For example, texture2D(sTexture2,texCoord) would become texture2D(sTexture,texCoord) and so forth. This will cause the same video to be applied multiple times to different geometry. The tiled geometry can also be changed by modifying gles2layout.java to suit your particular needs or you could replace the implementation altogether and simply provide your own vertex data. This would get the overlapping effect. Individual pixel processing algorithms & scaling operations / transforms could also be applied to the geometry or with fragement/vertex shader to achieve the other processing goals and may be targeted to specific geometry using the per vertex color as an indicator of what effect to apply. Another pretty simple approach would be through some creative use of texture coordinate mappings. Hope this helps and good luck!

Answer 2

Try using Google's ExoPlayer library . For the activity layout use a FrameLayout with two TextureViews, one large and one small with the smaller one on top to give the overlap effect.

ExoPlayer demo code is here .

Also ExoPlayer is a layer on top of MediaCodec (API 16), meaning this will work going back to JellyBean 4.1.

Update :

With both videos playing in the two TextureViews you can record the output to a video file.

High-level Overview:

(I'll try to give a better code sample later)

Initialization

• Prepare a Bitmap-based Canvas
• Prepare a MediaEncoder
• Prepare a MediaMuxer

In a periodic loop (ie once every 30 milliseconds):

• Draw root view to Canvas
• Enqueue bitmap to MediaEncoder
• Mux to MP4

Answer 3

To put multiple overlapped video frames on a testure you can use Multi-Media Sample Framework: https://software.intel.com/sites/landingpage/mmsf/documentation/mmsf_android_example2.html

To create video file from textures you can try to use GLCapture from Intel INDE Media for Mobile. Tutorials are here: https://software.intel.com/en-us/articles/intel-inde-media-pack-for-android-tutorials-video-capturing-for-opengl-applications and samples are here: https://github.com/INDExOS/media-for-mobile . The only issue is that GLCapture does not support audio from file, so if you need audio you might have to go through MediaCodec way.