Transmit The Images Numpy Array Faster in Python

Question

Currently, I've developed a simple Web Application using Flask Framework. The main language which I am using is Python.

I'm supposing that I have 2 primary files that are Flask Server and Flask Client.

The Flask Server will initialize a Deep Learning Model (eg Object detection).

The Flask Client will grab frames from the camera source and send the frames to Flask Server to get the bounding boxes of objects.

My question here is if the video frames are too large (4k resolution). It will take a lot of time to transfer the frames. The time would be 200 ms for only one frame.

I want to make a real-time application on the client-side. Has anyone known the good approach for passing frames from this Python application to another Python application? I only consider the progress is happening in the local.network (at the same computer).

Answer 1

Python 3.8 has exactly the thing for you - Shared Memory . You can create a lump of shared memory and access it from independent Python processes on the same machine. I made a very simple example with a single frame of 4k UHD video (3840x2160) shared between 2 separate Python processes. So, start this one ( shmemA.py ) in one Terminal, preferably in IPython:

#!/usr/bin/env python3

import numpy as np
from multiprocessing import shared_memory

# Define dimensions of 4k video
w, h = 3840, 2160

# Create a named SharedMemory object
shm = shared_memory.SharedMemory(create=True, name="SharedVideoBuffer", size=w*h*3)

# Create a Numpy array backed by that SharedMemory
im = np.ndarray((h,w,3), dtype=np.uint8, buffer=shm.buf)

# Fill that array such that the other process can see it
im[:] = 32

And start this ( shmemB.py ) in a separate Terminal afterwards:

#!/usr/bin/env python3

import numpy as np
from multiprocessing import shared_memory

# Define dimensions of 4k video
w, h = 3840, 2160

# Attach to existing SharedMemory created by our buddy
existing_shm = shared_memory.SharedMemory(name='SharedVideoBuffer')

# Create Numpy array backed by that SharedMemory
im = np.ndarray((h,w,3), dtype=np.uint8, buffer=existing_shm.buf)

You can now time how long it takes to write from the first process to the second with:

%timeit im[:]=56
776 µs ± 20.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

And you can see it takes 776 microseconds to transmit a 4K frame between the processes, so it could achieve over 1,000 fps.

In your scenario, you might consider double-buffering (ie ping-ponging) between two buffers so that one process is writing the second while the other is reading the first. So you would create 2 shared memory buffers (or a single one twice the size) and alternate between them. You could use a multiprocessing Queue between the two processes to tell the reader which buffer has just been filled.

Keywords : Python, shared memory, SharedMemory, fast IPC, ping-pong, double-buffered, fast image transfer.