How can I get frame by using Gstreamer?

Question

I`ma beginner at using Gstreamer to handle some input videos. I have already built the pipeline using GStreamer to transcode the videos but the last part I cannot do is How I can get those batches of frames and do some custom image processing techniques to handle the purpose of my task.

---

Input Videos -----> Gstreamer Pipeline -----> Task: Apply some Image Processing Techniques

I`ve been searching about this problem on the Inte.net but cannot find any solution and the more I search, the more I am confused.

Answer 1

AppSink is the good element for you. You can enable "emit-signal" property and listen the event "new-sample". Then you can get an access to the buffer.

Here the entire documentation:

https://gstreamer.freedesktop.org/documentation/tutorials/basic/short-cutting-the-pipeline.html?gi-language=c

You have to create appsink element, enable "emit-signals" then register "new-sample" callback like this:

g_signal_connect (data.app_sink, "new-sample", G_CALLBACK (new_sample), &data)

static GstFlowReturn new_sample (GstElement *sink, CustomData *data) {
  GstSample *sample;
  /* Retrieve the buffer */
  g_signal_emit_by_name (sink, "pull-sample", &sample);
  if (sample) {
    /* The only thing we do in this example is print a * to indicate a received buffer */
    g_print ("*");
    gst_sample_unref (sample);
    return GST_FLOW_OK;
  }
  return GST_FLOW_ERROR;
}

Now you can retrieve buffer from sample instead of g_print... (gst_sample_get_buffer)

https://gstreamer.freedesktop.org/documentation/gstreamer/gstsample.html?gi-language=c

Then read data inside the buffer:

  GstMapInfo info;
  gst_buffer_map (buf, &info, GST_MAP_READ);
  
  gst_buffer_unmap (buf, &info);
  gst_buffer_unref (buf);

info.data ==> buffer content.

Best regards.