如何简化 sql 查询？

Question

I have the below query which is not working in sql server with large amount of data. The query is not working when I use date filter with 3 combinations in where clause.

      IF OBJECT_ID('tempdb..#tempAllocStatus1') IS NOT NULL
                     DROP TABLE #tempAllocStatus1 

                     SELECT  Users.Name,REPLACE(Staff.Designation, 'IND ', '') as Designation, Staff.Office as Location,
                                    (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt  
                                    INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                                    WHERE ReqDt.AssignedToID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) [Indian Benchmarking Assigned], 

                                    (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt  
                                    INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                                    WHERE ReqDt.ReviewerID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) [Indian Benchmarking Reviewer], 

                                    (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt  
                                    INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                                    WHERE ReqDt.SignoffID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [Indian Benchmarking Signoff],

                                    (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt  
                                    INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                                    WHERE ReqDt.AssignedToID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) + 
                                    (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt  
                                    INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                                    WHERE ReqDt.ReviewerID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) + 
                                    (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt  
                                    INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                                    WHERE ReqDt.SignoffID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [Indian Benchmarking Total],


                       (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt 
                        WHERE ReqDt.AssignedToID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) [All Assigned], 
                        (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt 
                        WHERE ReqDt.ReviewerID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) [All Reviewer], 
                        (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt 
                        INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                        WHERE ReqDt.SignoffID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [All Signoff],
                        (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt 
                        WHERE ReqDt.AssignedToID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) + 
                        (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt 
                        WHERE ReqDt.ReviewerID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) + 
                        (SELECT  COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt 
                        INNER JOIN  tblTPRequest Req ON ReqDt.RequestID = Req.Id 
                        WHERE ReqDt.SignoffID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [All Total]

                        INTO #tempAllocStatus1

                        FROM tblUserRolesMapping Users

                            LEFT JOIN [tblstaff] Staff ON Users.UserADID = Staff.AD_Id                                  
                            LEFT JOIN tblTPRequestDetail UserAssigned ON UserAssigned.AssignedToID = Users.UserADID
                            LEFT JOIN tblTPRequestDetail UserReviewer ON UserReviewer.ReviewerID = Users.UserADID
                            LEFT JOIN tblTPRequestDetail UserSignoff ON UserSignoff.SignoffID = Users.UserADID

                            LEFT JOIN tblTPRequest TPRAssigned ON UserAssigned.RequestID = TPRAssigned.ID
                            LEFT JOIN tblTPRequest TPRReviewer ON UserReviewer.RequestID = TPRReviewer.ID
                            LEFT JOIN tblTPRequest TPRSignoff ON UserSignoff.RequestID = TPRSignoff.ID 

                        WHERE Users.Active = 1 and Users.[RoleId] !=6
                                  AND (TPRAssigned.crtddate >= '2017-04-01'  and TPRAssigned.crtddate<= '2017-05-30')

                                  OR (TPRReviewer.crtddate >= '2017-04-01'  AND TPRReviewer.crtddate<='2017-05-30')

                                  OR (TPRSignoff.crtddate >= '2017-04-01'  AND TPRSignoff.crtddate<='2017-05-30' )
                                   GROUP BY Users.UserADID,  Users.Name, Staff.Designation,Staff.Office 

                            SELECT * FROM #tempAllocStatus1 WHERE [All Total] > 0

                                UNION ALL

                            SELECT * FROM #tempAllocStatus1 WHERE [All Total] = 0 

And I am facing the issue in the below lines of the query.

                             AND (TPRAssigned.crtddate >= '2017-04-01'  and TPRAssigned.crtddate<= '2017-05-30')

                                  OR (TPRReviewer.crtddate >= '2017-04-01'  AND TPRReviewer.crtddate<='2017-05-30')


                                  OR (TPRSignoff.crtddate >= '2017-04-01'  AND TPRSignoff.crtddate<='2017-05-30')

I have tried many solutions to simplify this query. But nothing is working with large amount of data. It is processing for long time and getting timeout execution. Can anyone please help to simplify this query?

As per the suggestion, I have optimized query as below.

   select
    ReqDt.AssignedToID,
    COUNT(*) NumRecs 
into
    #tmpAssigned
from
    tblTPRequestDetail ReqDt
        INNER JOIN  tblTPRequest Req 
            ON ReqDt.RequestID = Req.Id
            AND Req.TypeOfRequest = 1 

            AND Req.crtddate >= '2017-04-01'  
            and Req.crtddate <= '2022-05-30'
WHERE

        ReqDt.StatusCode IN ( 'COMP', 'PCR' ) 
    OR (    ReqDt.StatusCode = 'AWIP' 
        and ReqDt.SubStatusCode IN ( 'BENRE', 'BENSF', 'DRR', 'DRSO', 'RPSOFF' )
       )
GROUP BY
    ReqDt.AssignedToID



    select
    ReqDt.ReviewerID,
    COUNT(*) NumRecs
into
    #tmpReviewed
from
    tblTPRequestDetail ReqDt
        INNER JOIN  tblTPRequest Req 
            ON ReqDt.RequestID = Req.Id
           AND Req.TypeOfRequest = 1 

           AND Req.crtddate >= '2017-04-01'  
           and Req.crtddate <= '2022-05-30'
WHERE

        ReqDt.StatusCode IN ( 'COMP', 'PCR' ) 
    OR (    ReqDt.StatusCode = 'AWIP' 
        and ReqDt.SubStatusCode IN ( 'BENSF', 'DRSO', 'RPSOFF', 'RPC', 'TPRPC' )
       )
GROUP BY
    ReqDt.ReviewerID 

    select
    ReqDt.SignoffID,
    COUNT(*) NumRecs 
into
    #tmpSigned
from
    tblTPRequestDetail ReqDt
        INNER JOIN  tblTPRequest Req 
            ON ReqDt.RequestID = Req.Id
            AND Req.TypeOfRequest = 1 

            AND Req.crtddate >= '2017-04-01'  
            and Req.crtddate <= '2022-05-30'
WHERE

        ReqDt.StatusCode IN ( 'COMP', 'PCR' ) 
    OR (    ReqDt.StatusCode = 'AWIP' 
        and ReqDt.SubStatusCode IN ( 'RPSOFF', 'SCPC', 'TPSCPC' )
        )
GROUP BY
    ReqDt.SignoffID


    SELECT U.Name,
    REPLACE(S.Designation, 'IND ', '') as Designation, 
    S.Office as Location,
    coalesce( Assigned.NumRecs, 0 ) [Indian Benchmarking Assigned],
    coalesce( Reviewed.NumRecs, 0 ) [Indian Benchmarking Reviewed],
    coalesce( Signed.NumRecs, 0 ) [Indian Benchmarking Signed],
    coalesce( Assigned.NumRecs, 0 ) + coalesce( Reviewed.NumRecs, 0 ) + coalesce( Signed.NumRecs, 0 )  [total Indian Benchmarking]

from
    tblUserRolesMapping U
        left JOIN [tblstaff] S 
            ON U.UserADID = S.AD_Id

        left JOIN #tmpAssigned Assigned
            on U.UserADID = Assigned.AssignedToID

        left JOIN #tmpReviewed Reviewed
            on U.UserADID = Reviewed.ReviewerID           

        left JOIN #tmpSigned Signed
            on U.UserADID = Signed.SignoffID

WHERE 
        U.Active = 1 
    and U.RoleId !=6 
    and coalesce( Assigned.NumRecs, 0 ) + coalesce( Reviewed.NumRecs, 0 ) + coalesce( Signed.NumRecs, 0 ) !=0

order by U.Name

But I am getting the duplicate records. How do I remove duplicate records from the result. Also the same user with USERADID can have more than one role in tblUserRoleMapping table.

Answer 1

Obviously redundant, but also consideration of your where conditions, especially in your counting queries. Looking at your (StatusCode and SubStatusCode or SubStatusCode or SubStatusCode or...). If any of your subsequent OR conditions are true it will be counted as true. Looking at StatusCode = 'AWIP' and SubStatus = 'DRSO', this would be interpreted as

( true and false or false or true or false ) resulting in TRUE

or another Looking at StatusCode = 'XYZ' and SubStatus = 'DRSO', this would be interpreted as

( false and false or false or true or false ) resulting in TRUE.

What I THINK you are trying to accomplish is (respectively per samples above

( true and (false or false or true or false )) resulting in TRUE
( false and (false or false or true or false )) resulting in FALSE.

where you want the primary status code = 'AWIP', and if so, then only count if any of the SUBSTATUS codes is any of the following. Your parenthesis were not accommodating correctly if this was intended. By changing to an IN () clause can help simplify this ambiguity.

Now, with the duplication, if this is pre-written as a sub-query, grouped by user ID and get the counts you can join to it once

Since you are already using "#" temp tables in SQL-Server, you can prequery these results and then sum() again without user ID for the grand-total per category.

Per comment of duplicate STAFF, you need to get the UNIQUE STAFF ID (not provided so I will make the column name up. You can obviously adjust that). Also, due to transitive association (if a=b and b=c then a=c) I will join to the STAFF table and get the STAFF unique ID. Also, by removing context of the "Role", I also made up a column for the StaffUserName. This will eliminate the duplicate records because it rolls-up the totals per PERSON, not per the persons ROLE.

select
        S.UniqueStaffID,
        COUNT(*) NumRecs 
    into
        #tmpAssigned
    from
        tblTPRequestDetail ReqDt
            JOIN [tblstaff] S 
                ON ReqDt.AssignedToID = S.AD_Id
            INNER JOIN  tblTPRequest Req 
                ON ReqDt.RequestID = Req.Id
                AND Req.TypeOfRequest = 1 
                -- apply the date filters directly to where you are getting queries 
                AND Req.crtddate >= '2017-04-01'  
                and Req.crtddate <= '2017-05-30'
    WHERE
            -- and only getting same criteria
            ReqDt.StatusCode IN ( 'COMP', 'PCR' ) 
        OR (    ReqDt.StatusCode = 'AWIP' 
            and ReqDt.SubStatusCode IN ( 'BENRE', 'BENSF', 'DRR', 'DRSO', 'RPSOFF' )
           )
    GROUP BY
        S.UniqueStaffID


select
        S.UniqueStaffID,
        COUNT(*) NumRecs
    into
        #tmpReviewed
    from
        tblTPRequestDetail ReqDt
            JOIN [tblstaff] S 
                ON ReqDt.AssignedToID = S.AD_Id
            INNER JOIN  tblTPRequest Req 
                ON ReqDt.RequestID = Req.Id
               AND Req.TypeOfRequest = 1 
               -- apply the date filters directly to where you are getting queries 
               AND Req.crtddate >= '2017-04-01'  
               and Req.crtddate <= '2017-05-30'
    WHERE
        -- and only getting same criteria
            ReqDt.StatusCode IN ( 'COMP', 'PCR' ) 
        OR (    ReqDt.StatusCode = 'AWIP' 
            and ReqDt.SubStatusCode IN ( 'BENSF', 'DRSO', 'RPSOFF', 'RPC', 'TPRPC' )
           )
    GROUP BY
        S.UniqueStaffID 


select
        S.UniqueStaffID,
        COUNT(*) NumRecs 
    into
        #tmpSigned
    from
        tblTPRequestDetail ReqDt
            JOIN [tblstaff] S 
                ON ReqDt.AssignedToID = S.AD_Id
            INNER JOIN  tblTPRequest Req 
                ON ReqDt.RequestID = Req.Id
                AND Req.TypeOfRequest = 1 
                -- apply the date filters directly to where you are getting queries 
                AND Req.crtddate >= '2017-04-01'  
                and Req.crtddate <= '2017-05-30'
    WHERE
            -- and only getting same criteria
            ReqDt.StatusCode IN ( 'COMP', 'PCR' ) 
        OR (    ReqDt.StatusCode = 'AWIP' 
            and ReqDt.SubStatusCode IN ( 'RPSOFF', 'SCPC', 'TPSCPC' )
            )
    GROUP BY
        S.UniqueStaffID



SELECT
        S.StaffUserName,
        REPLACE(S.Designation, 'IND ', '') as Designation, 
        S.Office as Location,
        coalesce( Assigned.NumRecs, 0 ) [Indian Benchmarking Assigned],
        coalesce( Reviewed.NumRecs, 0 ) [Indian Benchmarking Reviewed],
        coalesce( Signed.NumRecs, 0 ) [Indian Benchmarking Signed],
        coalesce( AllAssigned.NumRecs, 0 ) [All Indian Benchmarking Assigned],
        coalesce( AllReviewed.NumRecs, 0 ) [All Indian Benchmarking Reviewed],
        coalesce( AllSigned.NumRecs, 0 ) [All Indian Benchmarking Signed],
        coalesce( AllAssigned.NumRecs, 0 ) 
            + coalesce( AllReviewed.NumRecs, 0 ) 
            + coalesce( AllSigned.NumRecs, 0 ) [AllRecords]
    from
        JOIN [tblstaff] S 
            LEFT JOIN #tmpAssigned Assigned
                on S.UniqueStaffID = Assigned.UniqueStaffID
            JOIN
            ( select sum(NumRecs) NumRecs 
                from #tmpAssigned ) AllAssigned
                on 1=1

            LEFT JOIN #tmpReviewed Reviewed
                on S.UniqueStaffID = Reviewed.UniqueStaffID
            JOIN
            ( select sum(NumRecs) NumRecs 
                from #tmpReviewed ) AllReviewed
                on 1=1

            LEFT JOIN #tmpSigned Signed
                on S.UniqueStaffID = Signed.UniqueStaffID
            JOIN
            ( select sum(NumRecs) NumRecs 
                from #tmpSigned ) AllSigned
                on 1=1
    WHERE 
            U.Active = 1 
        and U.RoleId !=6
    ORDER BY 
        S.StaffUserName, 
        S.Designation,
        S.Office 

Since the pre-query of Assigned, Reviewed and Signed are pre-aggregated to at most, one user record at a time, you don't need to use a group by at the outer level. Also, for the "TOTAL" of assigned, reviewed, signed, since those are not grouped, they will always be a single record representing the entire qualified pool of records of respective categories. Should help you get your query done and totals as you were attempting.

Update per comment feedback.

I am not using any "distinct" in my query and you must have added that. As for why, I can only assume that a single user ROLE ID may be associated with multiple Staff? If you can edit your original post and share some SAMPLE Data representing such ROLES and STAFF might help (and do a couple roles/staff to see better correlation and obviously nothing truly private/confidential). This is probably the source of multiple records.

I was under the impression that the "UserADID" was a unique user, not a user ROLE as this ID was assigned to the request detail. I can only guess the impact but am now thinking you might have something like

ApprovingRole ID = 1
   Office 1 Staff person ID X
   Office 2 Staff Person ID Y
   Office 3 Staff Person ID Z

ReviewingRole ID = 2
   Office 2 Staff person ID A
   Office 2 Staff Person ID Y
   Office 4 Staff person ID B

SigningRole ID = 3
   Office 2 Staff Person ID Y
   Office 4 Staff person ID B

In the above "sample data" above, you can see that staff person "Y" is a member of all 3 roles. Staff person "B" is member of 2 roles.

etc

So which "ID" is really handled here. Is the Approving "Role" ID 1 tied to the request detail or is it the STAFF person X, Y, Z tied to.

If the data is as I have it, and from your lack of sample data, how would I know which staff site / designation the underlying counts associated with. The totals are otherwise representative of the ROLD User ID, not the STAFF User ID. Let me know otherwise and I can edit results.

So now you state there are duplicates for the same USER, not user ROLE, In that case. we need to have each prequery temp joined all the way to the USER STAFF ID. not the ROLE ID. I have modified the queries above.