[英]How do I serialize video frames for streaming over UDP?
I am trying out video streaming over UDP.我正在尝试通过 UDP 进行视频流传输。 I capture my screen using vidgear and use pickle for serialization.我使用 vidgear 捕获我的屏幕并使用 pickle 进行序列化。 I am trying to build a remote desktop solution therefore requiring low-latency, but I found that pickle is slow for the purpose.我正在尝试构建一个远程桌面解决方案,因此需要低延迟,但我发现泡菜的速度很慢。 Are there any other serialization frameworks that can serialize video frames?还有其他可以序列化视频帧的序列化框架吗? I was able to find flatbuffers and protobuf but I am not sure how to use these for video.我能够找到flatbuffers和protobuf ,但我不确定如何将它们用于视频。
So it would be greatly appreciated if someone could point me in the right direction, ie suggest a fast serialization framework.因此,如果有人能指出我正确的方向,我将不胜感激,即建议一个快速序列化框架。
Thanks in advance: :)提前致谢: :)
I see you already resolved it but meanwhile I made some example.我看到你已经解决了它,但同时我做了一些例子。
You can use tobytes()
to convert numpy.arraay
to bytes
which you can send by socket.您可以使用tobytes()
将numpy.arraay
转换为可以通过套接字发送的bytes
。
byte_data = arr.tobytes()
You can also use struct
to convert values its length to 4 bytes
or height,width,depth
to 12 bytes
您还可以使用struct
将其长度转换为4 bytes
或将height,width,depth
转换为12 bytes
size = len(byte_data)
byte_size = struct.pack('I', size)
width, height, depth = arr.shape
byte_width_height = struct.pack('III', width, height, depth)
And then you can send with size or with width, height, depth
然后你可以发送大小或width, height, depth
all_bytes = byte_size + byte_data
send(all_bytes)
all_bytes = byte_height_width + byte_data
send(all_bytes)
In client you can first get 4 bytes
with size在客户端你可以先得到4 bytes
的大小
byte_size = recv(4)
size = struct.unpack('I', byte_size)
or 12 byes
if you send it with height,width,depth
或12 byes
,如果您发送height,width,depth
byte_height_width_depth = recv(12)
height, width, depth = struct.unpack('III', byte_height_width_depth)
and then you know how may bytes has frame然后你知道字节有多少帧
byte_data = recv(size)
arr = np.frombuffer(byte_data, dtype=np.uint8)
with height,width,depth
you may know also how to reshape it使用height,width,depth
您可能还知道如何重塑它
byte_data = recv(height*width*depth)
arr = np.frombuffer(byte_data, dtype=np.uint8)
arr = arr.reshape((height, width, depth))
If you use frame always with the same height, width, depth
then you could send only data without height, width, depth
or even without `size and use hardcoded values in code.如果您始终使用具有相同height, width, depth
的框架,那么您可以只发送没有height, width, depth
甚至没有 `size 的数据,并在代码中使用硬编码值。
But if you plan to send it as compressed to JPG or PNG which may have different number of bytes then you will need to send size as first value.但是,如果您打算将其压缩为可能具有不同字节数的 JPG 或 PNG 发送,那么您需要将大小作为第一个值发送。
Using pickle
you get more bytes because it send information about class numpy.array
to reconstruct it.使用pickle
可以获得更多字节,因为它发送有关 class numpy.array
的信息来重建它。
Using tobytes
you have to reconstruct array on your own.使用tobytes
你必须自己重建数组。
Example code - it simulate to send
, recv
.示例代码 - 它模拟send
, recv
。
import numpy as np
import struct
import pickle
"""Simulater socket."""
internet = bytes()
pointer = 0
def send(data):
"""Simulater socket send."""
global internet
internet += data
def recv(size):
"""Simulater socket recv."""
global pointer
data = internet[pointer:pointer+size]
pointer += size
return data
def send_frame(arr):
#height, width, depth = arr.shape
#byte_height_width_depht = struct.pack('III', width, height, depth)
byte_height_width_depht = struct.pack('III', *arr.shape)
#send(byte_height_width_depht)
byte_data = arr.tobytes()
#send(byte_data)
all_bytes = byte_height_width_depht + byte_data
send(all_bytes)
print('all_bytes size:', len(all_bytes))
print('all_bytes data:', all_bytes)
def recv_frame():
byte_height_width_depht = recv(12)
height, width, depth = struct.unpack('III', byte_height_width_depht)
byte_data = recv(height*width*depth)
arr = np.frombuffer(byte_data, dtype=np.uint8).reshape((height, width, depth))
return arr
# --- main ---
arr = np.array([
[[255, 255, 255], [255, 255, 255]],
[[255, 0, 0], [ 0, 0, 255]],
[[255, 0, 0], [ 0, 0, 255]],
[[255, 255, 255], [255, 255, 255]],
], dtype=np.uint8)
print('--- pickle ---')
data = pickle.dumps(arr)
print('pickle size:', len(data))
print('pickle data:')
print(data)
print()
arr = pickle.loads(data)
print('array:')
print(arr)
print()
print('--- send frame ---')
send_frame(arr)
print()
print('--- recv frame ---')
arr = recv_frame()
print(arr)
print()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.