如何将子查询转换为连接以获得快速结果？

Question

I want to convert Sub Queries into joins to improve performance.

The following sub-queries take to long to load.

SELECT c.tank_name, c.fuel_type, c.capacity, c.tank_id,
     (SELECT TOP 1 b.Level 
      from Microframe.dbo.TrackMessages b 
      where b.IMEI = a.IMEI 
            AND b.Timestamp >= @Start 
      order by b.Timestamp ) AS Level,
    (select top 1 b.Timestamp 
     from Microframe.dbo.TrackMessages b 
     where b.IMEI = a.IMEI 
           AND b.Timestamp >= @Start 
     order by b.Timestamp ) AS TimeStamp,
    (SELECT top 1 b.Temp 
     from Microframe.dbo.TrackMessages b 
     where b.IMEI = a.IMEI 
           AND b.Timestamp >= @Start 
     order by b.Timestamp ) AS Temp 
FROM GatexServerDB.dbo.device as a
JOIN GatexReportsDB.dbo.tbl_static_tank_info as c ON c.tank_id = a.owner_id
WHERE c.client_id = 65
AND a.IMEI IS NOT NULL
AND c.tank_id IN ({Tanks})

Answer 1

You can move the subquery to the FROM clause and use CROSS APPLY . Since you seem to be dealing with IoT data though, you should investigate T-SQL's ranking, windowing and analytic functions. Performance will depend heavily on the table's indexes.

Given these tables :

create table #TrackMessages (
    Message_ID bigint primary key,
    imei nvarchar(50) ,
    [timestamp] datetime2,
    Level int,
    temp numeric(5,2)
);

create table #device (
    imei nvarchar(50) primary key,
    owner_id int        
);


create table #tbl_static_tank_info (
    tank_id int not null primary key,
    tank_name nvarchar(20),
    fuel_type nvarchar(20),
    capacity numeric(9,2),
    owner_id int,
    client_id int
 )

And indexes :

create nonclustered index IX_MSG_IMEI_Time on #TrackMessages (imei,timestamp) include(level,temp)       ;
create INDEX IX_Device_OwnerID on #device (Owner_ID)
create INDEX IX_Tank_Client on #tbl_static_tank_info (Client_ID);
create INDEX IX_Tank_Owner  on #tbl_static_tank_info (Owner_ID);

The TOP 1 query would look like this :

SELECT c.tank_name, c.fuel_type, c.capacity, c.tank_id,
     Level,
    TimeStamp,
    Temp 
FROM #device as a
inner JOIN #tbl_static_tank_info as c ON c.tank_id = a.owner_id
cross apply (SELECT top 1 imei,Temp,Level,timestamp 
            from #TrackMessages b 
            where b.IMEI = a.imei
           AND b.Timestamp >= @start 
     order by b.Timestamp ) msg 
WHERE c.client_id = 65
AND a.IMEI IS NOT NULL
AND c.tank_id IN (1,5,7)

If there is a 1-M relation between tanks, devices and messages, the FIRST_VALUE analytic function can be used to return the first record ber device, without using a subquery :

SELECT c.tank_name, c.fuel_type, c.capacity, c.tank_id,
        first_value(Temp) over (partition by b.imei order by timestamp) as temp,
        first_value(Level) over (partition by b.imei order by timestamp) as level,
        min(timestamp)  over (partition by b.imei) as timestamp
from #TrackMessages b 
    inner join #device as a on b.IMEI = a.imei
    inner JOIN #tbl_static_tank_info as c ON c.tank_id = a.owner_id
WHERE c.client_id = 65
AND a.IMEI IS NOT NULL
AND c.tank_id IN (1,5,7)

Performance will depend heavily on the indexes, the table statistics and whether the index and OVER order matches.

This query can be modified to return both the first and last value per device using LAST_VALUE :

SELECT c.tank_name, c.fuel_type, c.capacity, c.tank_id,
        first_value(Temp) over (partition by b.imei order by timestamp) as StartTemp,
        first_value(Level) over (partition by b.imei order by timestamp) as StartLevel,
        min(timestamp)  over (partition by b.imei) as StartTime,
        last_value(Temp) over (partition by b.imei order by timestamp) as EndTemp,
        lastt_value(Level) over (partition by b.imei order by timestamp) as EndLevel,
        max(timestamp)  over (partition by b.imei) as EndTime   
from #TrackMessages b 
    inner join #device as a on b.IMEI = a.imei
    inner JOIN #tbl_static_tank_info as c ON c.tank_id = a.owner_id
WHERE c.client_id = 65
AND a.IMEI IS NOT NULL
AND c.tank_id IN (1,5,7)

the server would have to sort the measurements both by ascending timestamp order (that's what the IX_MSG_IMEI_Time index already does) and descending order.

Answer 2

Here's a solution with CROSS APPLY which is like a function you can declare on the go and use it as a joining clause. You can change CROSS APPLY to OUTER APPLY if the returning set might not exist, in this case if there might not be any record on TrackMessages for a particular IMEI (will return NULL values).

SELECT 
    c.tank_name, 
    c.fuel_type, 
    c.capacity, 
    c.tank_id,

    T.Level,
    T.Timestamp,
    T.Temp

FROM GatexServerDB.dbo.device as a
JOIN GatexReportsDB.dbo.tbl_static_tank_info as c ON c.tank_id = a.owner_id
CROSS APPLY (

    SELECT TOP 1 -- Retrieve only the first record

        -- And return as many columns as you need
        b.Level,
        b.Timestamp,
        b.Temp
    FROM
        Microframe.dbo.TrackMessages AS b
    WHERE
        a.IMEI = b.IMEI AND -- With matching IMEI
        b.Timestamp >= @Start
    ORDER BY
        b.Timestamp) T -- Ordered by Timestamp

WHERE c.client_id = 65
AND a.IMEI IS NOT NULL
AND c.tank_id IN ({Tanks})

However I believe the key point here would be indexes on your tables. If you are already sure that the problem is the subquery, then make sure that TrackMessages has the following index:

CREATE NONCLUSTERED INDEX NCI_TrackMessages_IMEI_TimeStamp ON Microframe.dbo.TrackMessages (IMEI, Timestamp)

Indexes have pros and cons, make sure to check them out before creating or dropping one.

Answer 3

Not having the structures, my guees for the solution is:

WITH CTE AS 
    (SELECT B.IMEI,
            b.Level, 
            b.Timetamp,
            b.Temp,
            ROW_NUMBER() OVER (PARTITION BY b.IMEI ORDER BY Timestamp) AS Row
      FROM Microframe.dbo.TrackMessages b 
      WHERE b.Timestamp >= @Start 
    )
SELECT c.tank_name, c.fuel_type, c.capacity, c.tank_id,
    CTE.Level, CTE.Timestamp, CTE.Temp
FROM GatexServerDB.dbo.device as a
INNER JOIN GatexReportsDB.dbo.tbl_static_tank_info as c ON c.tank_id = a.owner_id
INNER JOIN CTE ON  CTE.IMEI = a.IMEI 
WHERE c.client_id = 65
  AND a.IMEI IS NOT NULL
  AND c.tank_id IN ({Tanks})
  AND CTE.Row = 1;

I cannot test it but it should be very close to the solution. Please, confirm if it works.

Answer 4

You can compare and go with either of the below solutions

The JOIN way where ordering is done via row-number windowing function

SELECT * FROM 
(
    SELECT 
        c.tank_name, 
        c.fuel_type, 
        c.capacity, 
        c.tank_id,
        Level=b.Level,
        TimeStamp=b.Timestamp,
        Temp=b.Temp,
        r=Row_number() over ( order by b.timestamp)
    FROM GatexServerDB.dbo.device as a
        JOIN GatexReportsDB.dbo.tbl_static_tank_info as c 
            ON c.tank_id = a.owner_id
        JOIN Microframe.dbo.TrackMessages as b 
            ON b.IMEI = a.IMEI AND b.Timestamp >= @Start 
    WHERE c.client_id = 65
    AND a.IMEI IS NOT NULL
    AND c.tank_id IN ({Tanks})
)T
where r=1

or the CROSS APPLY way like below

SELECT * FROM 
(
    SELECT 
        c.tank_name, c.fuel_type, c.capacity, c.tank_id
    FROM GatexServerDB.dbo.device as a
        JOIN GatexReportsDB.dbo.tbl_static_tank_info as c 
            ON c.tank_id = a.owner_id
            AND c.client_id = 65
            AND a.IMEI IS NOT NULL
            AND c.tank_id IN ({Tanks})
) A
CROSS APPLY 
(
    SELECT 
        TOP 1 
        b.Level, b.Timestamp,b.Temp 
    FROM Microframe.dbo.TrackMessages b
    WHERE b.IMEI = a.IMEI 
        AND b.Timestamp >= @Start 
    ORDER BY b.Timestamp 
)D