How to store the plant data and where to store?

Question

Plant data is real time data from plant process, such as, press, temperature, gas flow and so on. The data model of these data is typically like this:
(Point Name, Time stamps, value(float or integer), state(int))

We have thousands of points and longtime to store. And important, we want search them easy and quickly when we need.

A typically search request is like:

get data order by time stamp 
    from database 
    where Point name is P001_Press 
    between 2010-01-01 and 2010-01-02

A database similar to MySql is not suitable for us, because the records is too many and the query is too slowly.

So, how to store data (like above) and where to store them? Any NOSQL databases??

Thanks!

Answer 1

This data and query pattern actually fits pretty well into a flat table in a SQL database, which means implementing it with NoSQL will be significantly more work than fixing your query performance in SQL.

If your data is inserted in real time, you can remove the order by clause as the date will already be sorted by timestamp and there is no need to waste time resorting it. An index on point name and timestamp should get you good performance on the rest of the query.

If you are really getting to the limits of what a SQL table can hold (many millions of records) you have the option of sharding - a table for each data point may work fairly well.