Drawing bounding boxes on video streams using python

Question

I am using AWS Kinesis Video Streams to stream my camera on to the cloud. I then analyse it with AWS Lambda using Sagemaker, which returns the name of the objects and their coordinates in the video stream.

Now, I would like to improve my Lambda code (python) that creates bounding boxes on the kinesis video streams.

My problems are: 1. How do I grab the stream? 2. How do I overlap bounding boxes using coordinates? Do I use cv2? 3. How do I upload the stream to a new stream?

I saw a code doing this but in Java. Here's a link .

Answer 1

Here is my approach to your problems.

1. How do I grab the stream?

You can use aws boto3 library to get the video frames using the GET_MEDIA api or streams using KinesisVideoArchivedmedia to get the HLS or DASH streaming url which seems to solve your problem but GET_MEDIA api has less overheads. boto3 documentation

2. How do I overlap bounding boxes using coordinates? Do I use cv2?

You can use any video rendering library such as pyav or opencv to do it. The code to do it with opencv is easily available on the internet but you also need to consider other challenges like AWS Lambda limitations, consistency of available bounding boxes coordinates with the frame rate(you need atleast 3-4fps of object detections with a 15fps video to make it look real time)

3. How do I upload the stream to a new stream?

If you want to upload the stream back to Kinesis Video Stream, it is quite difficult and I don't think Lambdas will be sufficient for all the required processing. You may use ec2 instances for the purpose. Other option is to save the rendered video to s3 and have a video transcoding service(like AWS Mediaconvert, or build your own) to transcode the rendered video to HLS or DASH stream. The HLS or DASH stream saved on s3 can be streamed using content delivery network such as AWS Cloudfront.

Hope this helps.