Appropriate data structure for a buffer of the packets

Question

How to implement a buffer of the packets where each packet is of the form:

typedef struct{
   int32 IP;     //4-byte IP-address
   int16 ID;     //unique sequence id
}t_Packet;

What should be the most appropriate data structure which:

(1) allows to collect at least 8000 such packets (fast Insert and Delete operations)
(2) allows very fast filtering using IP address, so that only packets with given IP will be selected
(3) allows very fast find operation using ID as a key
(4) allows very fast (2), then (3) within filtered results ?

RAM size does matter, eg no huge lookup table is possible to use.

Answer 1

You can use a Patricia Trie for the IP address filtering. I believe most Network Routers use this data structure for IPV4 IP addresses. There are also other tries in literature designed for IP addresses which you can consider using. Here is one: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.4871

Correponding to each IP address, you can now have a hashtable based on ID.

If the IP addresses are 'sufficiently' random, you might be better off using a hashtable to do filtering based on IP, though, as the IP addresses nicely fit in a word of most machines, making the hash lookups really fast and the trie might not really save you much space.

Of course, the right choice depends on your situation...

Answer 2

Create two indices to the data (store it non-sequentially for fast insertions, etc), one a tree split by ID, the other a tree split by IP.

If you can't afford at least 8000 * sizeof(Packet) + 8000*12 + 8000*12 for data + two indices, then honestly, iterating over only 8000 items wouldn't really take very long.

This would be much easier to implement in c++ than c (if only because you could use boost::multi_index, or similar)