[英]Reusing Python Bytearray/Memoryview
I am attempting to receive a series of protobufs via a socket; 我试图通过套接字接收一系列protobuf。 I won't know the amount of data in advance.
我不会事先知道数据量。 I'm sending a fair amount of them, and need to buffer the messages as I receive them (to ensure that I get all messages).
我正在发送大量消息 ,并且需要在收到消息时对消息进行缓冲 (以确保我收到所有消息)。 I would like to make use of the bytearray/memoryview available in Python to eliminate an unnecessary copy.
我想利用Python中可用的bytearray / memoryview消除不必要的副本。
I'm currently using a string and appending the data as I receive it. 我目前正在使用字符串,并在收到数据时附加数据。 This is easy and I can "shift" down the "buffer" by doing something like:
这很容易,我可以通过执行以下操作来“下移”“缓冲区”:
# Create the buffer
str_buffer = []
# Get some data and add it to our "buffer"
str_buffer += "Hello World"
# Do something with the data . . .
# "shift"/offset the message by the data we processed
str_buffer = str_buffer[6:]
Is it possible to do something similar using a bytearray/memoryview? 是否可以使用bytearray / memoryview做类似的事情?
# Create the buffer/memoryarray
buffer = bytearray(1024)
view = memoryview(buffer)
# I can set a single byte
view[0] = 'a'
# I can "offset" the view by the data we processed, but doing this
# shrinks the view by 3 bytes. Doing this multiple times eventually shrinks
# the view to 0.
view = view[3:]
The problem arises when I try to add more data to the end. 当我尝试向末尾添加更多数据时,就会出现问题。 If I ever "offset" the existing view, the size of the view "shrinks*" and I can add less and less data.
如果我“偏移”了现有视图,则视图的大小“缩小”,并且我可以添加越来越少的数据。 Is there anyway to reuse the existing memoryview and just shift the data to the left?
无论如何,有没有要重用现有的memoryview并将数据向左移动?
*Based on the documentation, I know I can't resize the array. *根据文档,我知道我无法调整数组的大小。 I think the illusion of shrinking is a misunderstanding on my part.
我认为缩小的幻想对我来说是一种误解。
you really, honestly don't need to know in advance how much data to expect, just keep reading until you don't get any more data: 老实说,您真的不需要提前知道要期待多少数据,只需继续阅读直到没有更多数据为止:
import socket, sys
HOST = 'localhost' # The remote host
PORT = 50007 # The same port as used by the server
recvbuff = bytearray(16)
recvview = memoryview(recvbuff)
size = 0
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
while True:
nbytes = s.recv_into(recvview)
if not nbytes:
break
size += nbytes
recvview = recvview[nbytes:]
if not len(recvview):
print "filled a chunk", recvbuff
recvview = memoryview(recvbuff)
print 'end of data', recvbuff[:len(recvview)], size
s.close()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.