How to increase the speed of FER processing ( analyze ) for videos having high fps?

Question

I am using a video which has around 30000 frames, trying to use the below FER code for emotion recognition

The entire process is taking anywhere between 10-15 hrs just to analyze the video?

Is there a way to speed up the processing time or any other algorithm to detect facial emotion?

Here is the code:

from fer import Video
from fer import FER
import os
import sys
import pandas as pd

location_videofile = "/Users/Akash/Desktop/videoplayback.mp4"
input_video = Video(location_videofile)

processing_data = input_video.analyze(face_detector, display=False, frequency=5)

Tried adding the frequency paramter in the analyze function as well, but of no use since the processing time is pretty much the same, i am assuming it affects the output and not the analyze function

Answer 1

With the following answer I will give you several solutions that may or may not work with your particular video.

• The FER code relies on tensorflow and opencv for processing the data. Assuming a default installation of these packages through pip, tensorflow is already running on gpu (you may want to double check that), while opencv is not. Some of the functionalities of opencv can run on gpu and they may be the ones that FER is using: in this case, you may want to build the opencv package with GPU support (you can take a look here ).
• Another solution is to downsample the video-frames of you video by your own before supplying it to FER.
• Downsample each frame of the video in order to reduce the number of pixels in each frame. This may give a huge speed-up, if you can afford it (ie faces are occupying much of the screen and the number of frame pixels is relatively high)
• Multiprocessing. You could split the video in several mini-videos that you can analyse with multiple python processes . In my opinion, this is the cheapest and more reliable way to deal with the speed issue without loss in accuracy