How to encode camera frames into mp4 on android

Question

I take camera preview frames from android camera in 640x480 (sufficient to me) and do some modifications over them. But now I need to encode those to new MP4 file (with audio).

Is this some how possible? I can't use ffmpeg due to its not so good license, but I've found Stagefright framework which should be probably capable of doing that. But I did not find any sort of official documentation or tutorials to do such a thing I need to do.

Is there a way to do it with this framework please? I don't need codes, I would be very glad just for pointing me the right direction.

Answer 1

There is one scenario where the use-case described is realized. Consider a scenario where the Camera output is fed to an OpenGL library where some effects are applied on the preview frames which need to be recorded.

Well in this case, you can use the traditional MediaRecorder with GrallocSource instead of CameraSource . The setup would look like thus:

MediaRecorder is set up with the GrallocSource . The input surfaces for recording are provided by the Camera + OpenGL combined operation which implement a SurfaceTextureClient . A good example for this can be found in SurfaceMediaSource_test modules .

Answer 2

stagefright is quite good if you must support API 9 and higher. But this framework is not official, as you saw. You can use the sample code in platform/frameworks/av at your risk.

The google source includes CameraSource, which provides the camera frames directly to the encoder. While this approach may be much more efficient (the pixels are not copied to the user space at all), it does not allow manipulation. It is possible to modify the C++ source, but I strongly recommend to access the Camera in Java, and pass the preview frames via JNI to stagefrght (OpenMAX) encoder. On some devices, this may force you to use software encoder. You must convert the video frames to YUV planar format for the encoder. See libyuv for optimized converters.

If you can restrict your support to API 16 and higher, it is safer to use the official Java MediaCdec API .