[英]What python deque-like containers retain valid iterators during mutation?
I see here that collections.deque is not a great choice if I want to retain valid iterators while mutating the data structure.我在这里看到 collections.deque 不是一个很好的选择,如果我想在改变数据结构的同时保留有效的迭代器。 Are there any data structures in python's stdlib that preserve valid iterators while supporting these two operations:
python的stdlib中是否有任何数据结构在支持这两个操作的同时保留有效的迭代器:
Consider a class that generates a stream of data.考虑一个生成数据流的类。 There are multiple consumers that call
__iter__
on this class to get an iterator to consume data in-order.有多个消费者在这个类上调用
__iter__
来获得一个迭代器来按顺序消费数据。 For sake of simplicity, assume the __iter__
calls happen before any __next__
is called.为简单起见,假设
__iter__
调用发生在任何__next__
调用之前。
Suppose the stream is large enough that it could exceed reasonable memory limits.假设流足够大,可能会超出合理的内存限制。 A reasonable thing to do is to use a deque, so that when an iterator is at the end of the deque, we block to wait for new data and
append
it.一个合理的做法是使用双端队列,这样当迭代器在双端队列的末尾时,我们阻塞等待新数据并
append
它。 When all iterators have visited an element, we no longer need to keep it in memory, so we popleft
.当所有迭代器都访问过一个元素时,我们不再需要将它保存在内存中,所以我们
popleft
。
The std::deque
implementation in C++ uses circular list of buffers and allows us to have valid iterators upon these two operations. C++ 中的
std::deque
实现使用缓冲区的循环列表,并允许我们在这两个操作上拥有有效的迭代器。 Is there something like this in python as well? python中也有类似的东西吗?
itertools.tee
is designed for your use case exactly. itertools.tee
专为您的用例而设计。 Internally, it uses iterators pointing into a shared FIFO queue, but the data structure is internal to the tee
implementation (which is written in C, at least in CPython).在内部,它使用指向共享 FIFO 队列的迭代器,但数据结构是
tee
实现的内部结构(它是用 C 编写的,至少是用 CPython 编写的)。
Implementing a FIFO queue with iterators (your actual question) on top of tee
would probably be possible, but it's tricky, in part because the iterator interface specifies that no more values can ever appear after StopIteration
has been raised, while a typical FIFO can still be used after a failed pop.在
tee
顶部实现带有迭代器(您的实际问题)的 FIFO 队列可能是可能的,但它很棘手,部分原因是迭代器接口指定在引发StopIteration
后不会再出现更多值,而典型的 FIFO 仍然可以在弹出失败后使用。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.