How to improve a SQL timestamp range query?

Question

I have a table records having three fields:

id - the row id
value - the row value
source - the source of the value
timestamp - the time when the row was inserted (should this be a unix timestamp or a datetime?)

And I want to perform a query like this:

SELECT timestamp, value FROM records WHERE timestamp >= a AND timestamp <= b

However in a table with millions of records this query is super inefficient!

I am using Azure SQL Server as the DBMS. Can this be optimised?

If so can you provide a step-by-step guide to do it (please don't skip "small" steps)? Be it creating indexes, redesigning the query statement, redesigning the table (partitioning?)...

Thanks!

Answer 1

After creating an index on the field you want to search, you can use a between operator so it is a single operation, which is most efficient for sql.

SELECT XXX FROM ABC WHERE DateField BETWEEN '1/1/2015' AND '12/31/2015'

Also, in SQL Server 2016 you can create range indexes for use on things like time-stamps using memory optimized tables. That's really the way to do it.

Answer 2

I would recommend using the datetime, or even better the datetime2 data type to store the date data (datetime2 being better as it has a higher level of precision, and with lower precision levels will use less storage).

As for your query, based upon the statement you posted you would want the timestamp to be the key column, and then include the value. This is because you are using the timestamp as your predicate, and returning the value along with it.

CREATE NONCLUSTERED INDEX IX_Records_Timestamp on Records (Timestamp) INCLUDE (Value)

This being said, be careful of your column names. I would highly recommend not using reserved keywords for columns names as they can be a lot more difficult to work with.