快速查询慢速创建表

Question

I have an issue on creating tables by using select keyword (it runs so slow). The query is to take only the details of the animal with the latest entry date. that query will be used to inner join another query.

SELECT *
FROM amusementPart a 
INNER JOIN (
    SELECT DISTINCT name, type, cageID, dateOfEntry 
    FROM bigRegistrations
    GROUP BY cageID
) r ON a.type = r.cageID

But because of slow performance, someone suggested me steps to improve the performance. 1) use temporary table, 2)store the result and use it and join it the the other statement.

   use myzoo 
   CREATE TABLE animalRegistrations AS 
   SELECT DISTINCT name, type, cageID, MAX(dateOfEntry) as entryDate 
   FROM bigRegistrations
   GROUP BY cageID

unfortunately, It is still slow. If I only use the select statement, the result will be shown in 1-2 seconds. But if I add the create table, the query will take ages (approx 25 minutes)

Any good approach to improve the query time?

edit: the size of big registration table is around 3.5 million rows

Answer 1

This is a multipart question.

1. Use Temporary Table
2. Don't use Distinct - group all columns to make distinct (dont forget to check for index)
3. Check the SQL Execution plans

Answer 2

Can you please try the query in the way below to achieve The query is to take only the details of the animal with the latest entry date. that query will be used to inner join another query The query is to take only the details of the animal with the latest entry date. that query will be used to inner join another query , the query you are using is not fetching records as per your requirement and it will faster:

    SELECT a.*, b.name, b.type, b.cageID, b.dateOfEntry
FROM amusementPart a
INNER JOIN bigRegistrations b ON a.type = b.cageID
INNER JOIN (SELECT c.cageID, max(c.dateOfEntry) dateofEntry 
                FROM bigRegistrations c 
            GROUP BY c.cageID) t ON t.cageID = b.cageID AND t.dateofEntry = b.dateofEntry

Suggested indexing on cageID and dateofEntry

Answer 3

Here you are not creating a temporary table. Try the following...

CREATE TEMPORARY TABLE IF NOT EXISTS animalRegistrations AS 
SELECT name, type, cageID, MAX(dateOfEntry) as entryDate 
FROM bigRegistrations
GROUP BY cageID

Answer 4

Have you tried doing an explain to see how the plan is different from one execution to the next?

Also, I have found that there can be locking issues in some DB when doing insert(select) and table creation using select. I ran this in MySQL, and it solved some deadlock issues I was having.

SET SESSION TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;

Answer 5

The reason the query runs so slow is probably because it is creating the temp table based on all 3.5 million rows, when really you only need a subset of those, ie the bigRegistrations that match your join to amusementPart. The first single select statement is faster b/c SQL is smart enough to know it only needs to calculate the bigRegistrations where a.type = r.cageID.

I'd suggest that you don't need a temp table, your first query is quite simple. Rather, you may just need an index. You can determine this manually by studying the estimated execution plan, or running your query in the database tuning advisor. My guess is you need to create an index similar to below. Notice I index by cageId first since that is what you join to amusementParks, so that would help SQL narrow the results down the quickest. But I'm guessing a bit - view the query plan or tuning advisor to be sure.

CREATE NONCLUSTERED INDEX IX_bigRegistrations ON bigRegistrations
(cageId, name, type, dateOfEntry)

Also, if you want the animal with the latest entry date, I think you want this query instead of the one you're using. I'm assuming the PK is all 4 columns.

SELECT name, type, cageID, dateOfEntry 
FROM bigRegistrations BR
WHERE BR.dateOfEntry =
    (SELECT MAX(BR1.dateOfEntry)
    FROM bigRegistrations BR1
    WHERE BR1.name = BR.name
    AND BR1.type = BR.type
    AND BR1.cageID = BR.cageID)