Detect when a new frame is displayed from Camera Stream using DirectShow

Question

I use the following code to connect to the device and stream the video, its works great except I do not know how to detect when the actual frame changes

vars

fGraph: iGraphBuilder;
fBuilder: iCaptureGraphBuilder2;
fDevEnum: iCreateDevEnum;
fClassEnum: iEnumMoniker;
fMoniker: iMoniker;
fSrc: iBaseFilter;
fFetched: pLongInt;
fvideoWindow: iVideoWindow;
fmediaControl: iMediaControl;
fEvent: IMediaEvent;
fMediaEvent: IMediaEventEx;

Code:

fGraph := createComObject(CLSID_FilterGraph) as iGraphBuilder;
fBuilder := createComObject(CLSID_CaptureGraphBuilder2) as iCaptureGraphBuilder2;
fBuilder.SetFiltergraph(fGraph);
fDevEnum := createComObject(CLSID_SystemDeviceEnum) as iCreateDevEnum;
fDevEnum.createClassEnumerator(CLSID_VideoInputDeviceCategory , fClassEnum , 0);
fClassEnum.next(1 , fMoniker , fFetched);
fMoniker.bindToObject(nil , nil , IID_IBaseFilter , fSrc);
fGraph.addFilter(fSrc , 'Video Capture');
fGraph.queryInterface(IID_IMediaControl , fmediaControl);
fGraph.queryInterface(IID_IVideoWindow , fvideoWindow);
fGraph.queryInterface(IID_IMediaEvent , fEvent);
fEvent.queryInterface(IID_IMediaEventEx , fMediaEvent); // TForm(fOwner)
fMediaEvent.SetNotifyWindow(self.Handle , WM_MMNOTIFY , Integer(self));
fBuilder.renderStream(@PIN_CATEGORY_PREVIEW , @MEDIATYPE_VIDEO , fSrc , nil , nil);
fvideoWindow.put_windowStyle(WS_CHILD or WS_CLIPSIBLINGS);
fvideoWindow.setWindowPosition(0 , 0 , self.width , self.height);
fvideoWindow.put_owner(self.Handle);
fmediaControl.run;

As you can see I tried using IMediaEvent and IMediaEventEx, but that only fires when the video stream starting or stopping occur, i need to know every time a new frame is added.

Answer 1

Video renderers do not report frame presentation such as using a callback. When frame reaches video renderer, it is queued for presentation according to a time stamp possibly attached. Then video renderer decides on whether to discard, present or enqueue the data.

What you might want to do if you absolutely need to know presentation time, is to

• have a Sample Grabber filter connected before the renderer
• make sure that data is decompressed
• strip frame's time by removing respective flags in SampleCB grabber callback

The frame will be presented shortly, as soon as video renderer receives it. You still have control before it happens though. If you need it after , you need a custom filter that actually delivers the media sample to renderer and on returning control to your filter you know that the frame was just presented to the user.