简体   繁体   中英

In-memory Cache - SQLite vs System.Data.DataTable

I am at the very begining stage of designing an in-memory cache in C# (which will run as a windows service). Once in production, this is expected to hold close to a million objects (various types) on average. Some cache items can be up to 10MB (or more) in size. I considered a variety of data sotrage solutions and i have now decided to go for either a DataTable or an SQLite in-memory instance as the cache store. At this point my questions are

  1. How do you think the performance of a DataTable will be with this many number of records?
  2. Do you think going with an SQLite solution is an overkill?(Since SQLite is designed as a 'database', i may not really want all those database related plumbing activities)

Performance is the highest priority for me.

EDIT

Adding some more specifics.

  1. These cache items are not just key-value pairs, they have two more (as of now) properties (pinned and locked items), which can affect their availability. Every look up is going to include all the three properties.
  2. Memcached has been considered, but at this point that is not an option mainly due to our SLA constraints (That's all I can say about it).
  3. Not all items are of 10MB in size.
  4. I am pretty sure that many of these items are going to be mere numerical and small string values.
  5. I believe, availability of RAM is not an issue.

Thanks in advance, James

1: TERRIBLE. DataTbles are slow and Memory hogs, that wont magically Change for large items.
2: You tell us.

Have you considered using a simple dictionary? Key/Value pairs, you know.

The answers really dpend on what you plan doing with the Cache.

If every item is 1 MB that is 1 TB of memory.
You have 1 TB of memory to dedicate to this?
A database on a solid state disk may be a better design.

DataTable is large and slow.

How are you going to look the items up?
Are you going to have a complete key?
Are you going to have enough memeory?
If so dictionary.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM