How to get frame by fragmentNumberString from Kinesis Video Stream in Lambda?

Question

I am using Kinesis and Lambda. Kinesis Video Stream is processed using SageMaker endpoint for ML, output data is written to Kinesis Data Stream and then output data is processed in Lambda. If specific condition occurs, I need to visualize that frame and send image data, however, I am not able to retrieve that frame. I have fragment and frame metadata, for example:

FragmentMetadata(fragmentNumberString=111, serverSideTimestampMillis=1561970550674, producerSideTimestampMillis=1561970547121, fragmentNumber=111, success=true, errorId=0, errorCode=null, millisBehindNow=OptionalLong[2109], continuationToken=Optional[111])
Frame(trackNumber=1, timeCode=0, keyFrame=true, invisible=false, discardable=false, lacing=NO)

Is there a way to get image by fragmentNumberString from Kinesis Video Stream using Python?

Answer 1

There is no specific way to get a single frame/image using fragment number. Fragments are the smallest self contained processing unit in Kinesis Video which can contain a collection of frames. You must use any of the playback/retrieval API to get the fragments as whole, parse the MKV fragment into frames and process the frame/image. Some examples on how to do that:

GetMediaForFragmentList API: https://docs.aws.amazon.com/de_de/cli/latest/reference/kinesis-video-archived-media/get-media-for-fragment-list.html

Lambda example which gets fragments from KinesisVideo, extracts frames and overlays bounding boxes and creates a derivative stream: https://github.com/aws/amazon-kinesis-video-streams-parser-library#kinesisvideorekognitionlambdaexample