Python - Ordered List of Dictionaries, With a Size Limit?

Question

This is a requirement for a chat application project that I'm creating with Python, Flask, Socketio, and Javascript:

"Messages View: Once a channel is selected, the user should see any messages that have already been sent in that channel, up to a maximum of 100 messages. Your app should only store the 100 most recent messages per channel in server-side memory."

So, what I think I should be doing is creating a list of dictionaries. Like this:

messages = [ {"user":"Dave", "message":"Hello", "time":"12-24-2018"}, {"user":"John", "message":"Test", "time":"12-21-2018"} ]

My questions are..

1. How do I append to/access a list of dictionaries like this?
2. How do I limit the size of the list, and replace the oldest element with the second oldest element as I add to it?
3. Is this the best practice for storing a fixed size of data server side?

Answer 1

One possible solution is to use the double-ended queue implemented in the standard library's collections package - collections.deque .

Deques are similar to lists, but support efficient appending and popping from both ends, are threadsafe, and can be specified to have a maximum length.

For example:

>>> dq = collections.deque(maxlen=5)
>>> for i, x in enumerate('abcde'):
...    dq.append({x: i})
... 
>>> dq
deque([{'a': 0}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4}], maxlen=5)

If you add an element to a deque which has maxlen set, and which is already at maximum size, an element is removed from the other end of the deque:

>>> dq.append({'f': 5})
>>> dq
deque([{'b': 1}, {'c': 2}, {'d': 3}, {'e': 4}, {'f': 5}], maxlen=5)

>>> dq.appendleft({'z': 25})   
>>> dq
deque([{'z': 25}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4}], maxlen=5)

It's worth noting that Python's json module doesn't handle deques, so if you want to dump the deque to json you'll need to provide a function that converts the deque to a list:

>>> json.dumps(dq)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  ...
TypeError: Object of type deque is not JSON serializable
>>> def jdq(o):
...     return list(o)
... 
>>> json.dumps(dq, default=jdq)
'[{"z": 25}, {"b": 1}, {"c": 2}, {"d": 3}, {"e": 4}]'

To recreate the deque from a json array, just pass the deserialised list to a new deque:

>>> L = json.loads('[{"z": 25}, {"b": 1}, {"c": 2}, {"d": 3}, {"e": 4}]')
>>> dq = collections.deque(L, maxlen=5)
>>> dq
deque([{'z': 25}, {'b': 1}, {'c': 2}, {'d': 3}, {'e': 4}], maxlen=5)