4 million rows of data in a table

Question

I have 4 million rows of data in a table and I run this query

Select * from B

When I check the cpu usage, this query gets high cpu usage. My question is how can I improve cpu usage in this SQL query?

Answer 1

Are you likely to read a report with four million rows in it? I certainly wouldn't.

And, if not, why are you generating it?

If you're dumping the entire table for something like backup purposes, there are probably better ways, specific to the DBMS you're using.

If you don't need all the data (or even all of it at once), anything that can be used to limit what's coming out (such as a where clause to limit rows and/or more selective column selection than select * ) should help you out, as will proper indexing so the conditions in the where clause can be sped up. This is especially true if the data is going "across the wire", you don't want to be sending unneeded gigabytes across the network.

If you really want all columns from all four million rows in normal output format at one time, you'll just have to suffer the performance hit. Databases offer all sorts of ways to efficiently get at data but, if you want the lot, there's not much they can do.

Having said that, there are way to mitigate the impact but it depends on how you have things set up. Some examples are:

• Have your database replicated, using the primary copy for its intended purpose and a slave replica for reporting. Then hitting the replica won't affect the primary one.
• If you can execute a number of smaller queries rather than one big one to get the same result, that's a possibility. For example one query to get all records from 2015, another after some rest time to get the 2014 ones and so on.

That's two things off the top of my head, no doubt there are others but, without knowing more detail, it's hard to advise specifics.

Answer 2

Quite simply reduce what your generating.

Limit the selections

SELECT TOP (100) * FROM B

Include a where clause

SELECT * FROM B WHERE COLA = XXX AND COLB = YYY

Specify the columns

SELECT COLA, COLB, COLC, COLD FROM B WHERE COLA = XXX AND COLB = YYY

OR, if you must return all 4 million rows.

Create a view that populates a temp table and set up a job with it in which is ran during "down time", middle of the night etc, then select from the temp table