我如何存储实时数据

Question

I am developing a stock market application in windows form using .net framework 4.0. I am fetching ticks from the exchange real-time. I am able to receive the ticks on real-time basis. In my scenario I have to create a thread which will write the real-time feeds in generic collection & other continuously running background thread will fetch it from the collection & does further processing. To achieve this i am using the combination of ConcurrentDictionary and ConcurrentQueue . Key conatins symbol name & value contains ConcurrentQueue. This is how it has been implemented Check first answer

This is working just fine my only concern is the delay. This process is causing 1 second delay as stocks prices have to be fluctuated on real-time basis. Can this scenario be implemented by some other logic or can i improve the performance of ConcurrentDictionary or ConcurrentQueue somehow in order to avoid delays?

Answer 1

Do you really need a concurrent dictionary? Your dictionary changes only when you find a new symbol. If you happen to know all the symbols in advance, you can create an ordinary dictionary, fill it with a queue per symbol and that's it - the dictionary never changes beyond that.

I'm not sure how your processing thread works, though - how does it know which queue to look at next? Does it loop over all the queues to see if there is some more data? Perhaps what you need is just one ConcurrentQueue that will hold all the stock market events, and divide it into separate containers in the background there (perhaps with no synchronization).

Answer 2

Why don't you Enqueue an object which contains stmbol-id and corresponding Page, Dequeue it by the worker threads (without assigning a dedicated thread to each symbol id), and then, the worker thread will send the page to the correct processing depending on the symbol id by using a simple switch case statement.

this way, you don't have to check whether a Queue exists for a symbol. You'll only have a single thread-safe Queue of type AND you could use the ThreadPool for the processing!

the worker threads will do this:

IdPagePair pair = market.Dequeue();

if (pair.Id == 1) { Process1(pair.Page); }

else 

if (pair.Id == 2) { Process2(pair.Page); }

// etc...