Notify parent on change in child

Question

I have a .NET application that manages several large collections of items. Each item has a "scan time" interval (every 1 second, every 5 seconds, every minute and so on, it is user defined). When it is time, the item needs to run an internal method that might change its state and write to the DB.

At the moment the application structure is basically:
Each collection is managed in a different thread, and in each thread there is a parallel for each loop that goes over all of the items and for each item it checks if enough time has passed and the item needs to run his internal method. This approach causes issues when there is a big collection to one or more of the threads, with many items that has a small "scan time". Since it goes over all of the items in order to find out the ones that needs to be updated.

I thought to change the structure to some sort of Observer pattern. My idea is the each item will run a timer and notify the parent thread that it needs to be updated. I saw that there is a .NET library called Reactive Frameworks but I am not sure if I can use it for this.

Any other ideas on how to improve the current process are welcome.

Answer 1

I think you should always iterate on all items to find-out timed-out items that needs processing. Making a timer for each item is a very bad idea as your items are very much.

Suggestions:

1- Do not make multiple loops for checking scan-period time-out. Just make a single loop for checking all items.

2- If an item needs processing due to its time-out, put it in a queue for processing. You can process for example 4 items each time from queue. This will enhance performance, since a too extra multi-threading comes with opposite retsults.

3- You can try grouping your item based on their scan-period. This will fasten search because you will not have to search for every item, but for every group.