Effective queue/linked list in Python

Question

I need to write event calendar in Python which allows pasting events in any position AND works as FIFO (pop elements from the left side).

Python collections.deque can efficiently works as FIFO, but it don't allow to paste elements between current elements.

In the other hand, Python list allows inserting into the middle, but popleft is inefficient.

So, is there some compromise?

UPD Such structure probably more close to linked list than queue. Title changed.

Answer 1

You can have a look at blist . Quoted from their website:

The blist is a drop-in replacement for the Python list the provides better performance when modifying large lists.

...

Here are some of the use cases where the blist asymptotically outperforms the built-in list:

Use Case                                      blist              list
--------------------------------------------------------------------------
Insertion into or removal from a list         O(log n)           O(n)
Taking slices of lists                        O(log n)           O(n)
Making shallow copies of lists                O(1)               O(n)
Changing slices of lists                      O(log n + log k)   O(n+k)
Multiplying a list to make a sparse list      O(log k)           O(kn)
Maintain a sorted lists with bisect.insort    O(log**2 n)        O(n)

Some performance numbers here --> http://stutzbachenterprises.com/performance-blist

Answer 2

It's a bit of a hack but you can also use the SortedListWithKey data type from the SortedContainers module. You simply want the key to return a constant so you can order elements any way you like. Try this:

from sortedcontainers import SortedListWithKey

class FastDeque(SortedListWithKey):
    def __init__(self, iterable=None, **kwargs):
        super(FastDeque, self).__init__(iterable, key=lambda val: 0, **kwargs)

items = FastDeque('abcde')

print items
# FastDeque(['a', 'b', 'c', 'd', 'e'], key=<function <lambda> at 0x1089bc8c0>, load=1000)

del items[0]
items.insert(0, 'f')

print list(items)
# ['f', 'b', 'c', 'd', 'e']

The FastDeque will efficiently support fast random access and deletion. Other benefits of the SortedContainers module: pure-Python, fast-as-C implementations, 100% unit test coverage, hours of stress testing.

Answer 3

Just an idea - you can use heapq to maintain you list of events. As priority/keys for elements in a heap you can use events' timestamps.