选择正确的分类集合

Question

I am a bit in doubt on which collection to use for our data.

The domain is this (example):

For each supermarket we add a new item to a collection with a timestamp and total amount each time any customer pays at the register.

We currently do this:

We have a Dictionary collection with key = UniqueSupermarketID and value is a List<{timestamp, amount}>

Each time a customer pays we simply add a new item to the collection for the specific supermarket.

We need to extract data from this dictionary in a way that:

For a specified supermarket, get the newest cash register object with timestamp equaling "some timestamp"

We currently do this as:

supermarketDictionary["supermarket_01"]
    .OrderByDescending(i => t.TimeStamp)
    .FirstOrDefault(i => i.TimeStamp == 'some timestamp')

This obviously quickly starts performing like crap - so I am trying to figure out which collection to store data in instead.

I am considering using a normal dictionary to hold the "supermarket id <-> cash register list" relation and using a SortedDictionary for the timestamp/amounts used as keys .

Is this the correct approach? I would of course need to implement IComparable correctly on the timestamp to get it to work right.

Update 2014-01-03:

There are currently about 7 million rows in the list in question. The usages of the lists in our system have been identified as these:

_states
    .OrderBy(x => x.TimeStamp)
    .FirstOrDefault(x => x.WtgId == wtgId && x.IsAvailable && x.TimeStamp >= timeStamp);

_states
    .Where(x => x.WtgId == wtgId && x.IsAvailable && x.TimeStamp >= timeStamp && x.TimeStamp <= endDateTime)
    .OrderBy(x => x.TimeStamp).ToList();

_states.Remove(state);

if (!_states.Contains(message))
    _states.Add(message);

Thanks,

/Jesper Copenhagen, Denmark

Answer 1

EDIT: based on the update

All right, seeing what you really need sure helps to make a right decision. If your data comes already in order there is no need for a sorted collection and your four usages can be reduced to one ->

Searching for one item that matches some criteria

• adding with an existence check - adding is a cheap operation in non-sorted collections and existence check is just a searching for one item
• removing by item is also at the most one passing through a collection plus the remove operation itself which is also quite cheap (not in an array if done many times, though)

Try using PLINQ and carefully measure how it performs against LINQ. With so many entries, the difference should be nice.

_states.AsParallel().FirstOrDefault(...);

It will just create a few threads on the background and each of them will search some part of the collection and at the end results are merged. The .NET framework should choose the best number of threads for you, but if you feel like trying, apped .WithDegreeOfParallelism(x) where x is a number of threads it will use.