How can I simplify the sql query?
Question
I have the below query which is not working in sql server with large amount of data. The query is not working when I use date filter with 3 combinations in where clause.
IF OBJECT_ID('tempdb..#tempAllocStatus1') IS NOT NULL
DROP TABLE #tempAllocStatus1
SELECT Users.Name,REPLACE(Staff.Designation, 'IND ', '') as Designation, Staff.Office as Location,
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.AssignedToID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) [Indian Benchmarking Assigned],
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.ReviewerID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) [Indian Benchmarking Reviewer],
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.SignoffID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [Indian Benchmarking Signoff],
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.AssignedToID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) +
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.ReviewerID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) +
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.SignoffID = Users.UserADID AND Req.TypeOfRequest = 1 AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [Indian Benchmarking Total],
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
WHERE ReqDt.AssignedToID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) [All Assigned],
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
WHERE ReqDt.ReviewerID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) [All Reviewer],
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.SignoffID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [All Signoff],
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
WHERE ReqDt.AssignedToID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENRE' or ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRR' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF'))) +
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
WHERE ReqDt.ReviewerID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='BENSF' or ReqDt.SubStatusCode='DRSO' or ReqDt.SubStatusCode='RPSOFF' or SubStatusCode='RPC' or SubStatusCode='TPRPC'))) +
(SELECT COUNT(ReqDt.ID) FROM tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req ON ReqDt.RequestID = Req.Id
WHERE ReqDt.SignoffID = Users.UserADID AND ((ReqDt.StatusCode = 'COMP' or ReqDt.StatusCode = 'PCR') or (ReqDt.StatusCode = 'AWIP' and ReqDt.SubStatusCode='RPSOFF' or ReqDt.SubStatusCode='SCPC' or ReqDt.SubStatusCode='TPSCPC'))) [All Total]
INTO #tempAllocStatus1
FROM tblUserRolesMapping Users
LEFT JOIN [tblstaff] Staff ON Users.UserADID = Staff.AD_Id
LEFT JOIN tblTPRequestDetail UserAssigned ON UserAssigned.AssignedToID = Users.UserADID
LEFT JOIN tblTPRequestDetail UserReviewer ON UserReviewer.ReviewerID = Users.UserADID
LEFT JOIN tblTPRequestDetail UserSignoff ON UserSignoff.SignoffID = Users.UserADID
LEFT JOIN tblTPRequest TPRAssigned ON UserAssigned.RequestID = TPRAssigned.ID
LEFT JOIN tblTPRequest TPRReviewer ON UserReviewer.RequestID = TPRReviewer.ID
LEFT JOIN tblTPRequest TPRSignoff ON UserSignoff.RequestID = TPRSignoff.ID
WHERE Users.Active = 1 and Users.[RoleId] !=6
AND (TPRAssigned.crtddate >= '2017-04-01' and TPRAssigned.crtddate<= '2017-05-30')
OR (TPRReviewer.crtddate >= '2017-04-01' AND TPRReviewer.crtddate<='2017-05-30')
OR (TPRSignoff.crtddate >= '2017-04-01' AND TPRSignoff.crtddate<='2017-05-30' )
GROUP BY Users.UserADID, Users.Name, Staff.Designation,Staff.Office
SELECT * FROM #tempAllocStatus1 WHERE [All Total] > 0
UNION ALL
SELECT * FROM #tempAllocStatus1 WHERE [All Total] = 0
And I am facing the issue in the below lines of the query.
AND (TPRAssigned.crtddate >= '2017-04-01' and TPRAssigned.crtddate<= '2017-05-30')
OR (TPRReviewer.crtddate >= '2017-04-01' AND TPRReviewer.crtddate<='2017-05-30')
OR (TPRSignoff.crtddate >= '2017-04-01' AND TPRSignoff.crtddate<='2017-05-30')
I have tried many solutions to simplify this query. But nothing is working with large amount of data. It is processing for long time and getting timeout execution. Can anyone please help to simplify this query?
As per the suggestion, I have optimized query as below.
select
ReqDt.AssignedToID,
COUNT(*) NumRecs
into
#tmpAssigned
from
tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req
ON ReqDt.RequestID = Req.Id
AND Req.TypeOfRequest = 1
AND Req.crtddate >= '2017-04-01'
and Req.crtddate <= '2022-05-30'
WHERE
ReqDt.StatusCode IN ( 'COMP', 'PCR' )
OR ( ReqDt.StatusCode = 'AWIP'
and ReqDt.SubStatusCode IN ( 'BENRE', 'BENSF', 'DRR', 'DRSO', 'RPSOFF' )
)
GROUP BY
ReqDt.AssignedToID
select
ReqDt.ReviewerID,
COUNT(*) NumRecs
into
#tmpReviewed
from
tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req
ON ReqDt.RequestID = Req.Id
AND Req.TypeOfRequest = 1
AND Req.crtddate >= '2017-04-01'
and Req.crtddate <= '2022-05-30'
WHERE
ReqDt.StatusCode IN ( 'COMP', 'PCR' )
OR ( ReqDt.StatusCode = 'AWIP'
and ReqDt.SubStatusCode IN ( 'BENSF', 'DRSO', 'RPSOFF', 'RPC', 'TPRPC' )
)
GROUP BY
ReqDt.ReviewerID
select
ReqDt.SignoffID,
COUNT(*) NumRecs
into
#tmpSigned
from
tblTPRequestDetail ReqDt
INNER JOIN tblTPRequest Req
ON ReqDt.RequestID = Req.Id
AND Req.TypeOfRequest = 1
AND Req.crtddate >= '2017-04-01'
and Req.crtddate <= '2022-05-30'
WHERE
ReqDt.StatusCode IN ( 'COMP', 'PCR' )
OR ( ReqDt.StatusCode = 'AWIP'
and ReqDt.SubStatusCode IN ( 'RPSOFF', 'SCPC', 'TPSCPC' )
)
GROUP BY
ReqDt.SignoffID
SELECT U.Name,
REPLACE(S.Designation, 'IND ', '') as Designation,
S.Office as Location,
coalesce( Assigned.NumRecs, 0 ) [Indian Benchmarking Assigned],
coalesce( Reviewed.NumRecs, 0 ) [Indian Benchmarking Reviewed],
coalesce( Signed.NumRecs, 0 ) [Indian Benchmarking Signed],
coalesce( Assigned.NumRecs, 0 ) + coalesce( Reviewed.NumRecs, 0 ) + coalesce( Signed.NumRecs, 0 ) [total Indian Benchmarking]
from
tblUserRolesMapping U
left JOIN [tblstaff] S
ON U.UserADID = S.AD_Id
left JOIN #tmpAssigned Assigned
on U.UserADID = Assigned.AssignedToID
left JOIN #tmpReviewed Reviewed
on U.UserADID = Reviewed.ReviewerID
left JOIN #tmpSigned Signed
on U.UserADID = Signed.SignoffID
WHERE
U.Active = 1
and U.RoleId !=6
and coalesce( Assigned.NumRecs, 0 ) + coalesce( Reviewed.NumRecs, 0 ) + coalesce( Signed.NumRecs, 0 ) !=0
order by U.Name
But I am getting the duplicate records. How do I remove duplicate records from the result. Also the same user with USERADID can have more than one role in tblUserRoleMapping table.
1 answers
solution1
0 2019-09-25 12:22:15
Obviously redundant, but also consideration of your where conditions, especially in your counting queries. Looking at your (StatusCode and SubStatusCode or SubStatusCode or SubStatusCode or...). If any of your subsequent OR conditions are true it will be counted as true. Looking at StatusCode = 'AWIP' and SubStatus = 'DRSO', this would be interpreted as
( true and false or false or true or false ) resulting in TRUE
or another Looking at StatusCode = 'XYZ' and SubStatus = 'DRSO', this would be interpreted as
( false and false or false or true or false ) resulting in TRUE.
What I THINK you are trying to accomplish is (respectively per samples above
( true and (false or false or true or false )) resulting in TRUE
( false and (false or false or true or false )) resulting in FALSE.
where you want the primary status code = 'AWIP', and if so, then only count if any of the SUBSTATUS codes is any of the following. Your parenthesis were not accommodating correctly if this was intended. By changing to an IN () clause can help simplify this ambiguity.
Now, with the duplication, if this is pre-written as a sub-query, grouped by user ID and get the counts you can join to it once
Since you are already using "#" temp tables in SQL-Server, you can prequery these results and then sum() again without user ID for the grand-total per category.
Per comment of duplicate STAFF, you need to get the UNIQUE STAFF ID (not provided so I will make the column name up. You can obviously adjust that). Also, due to transitive association (if a=b and b=c then a=c) I will join to the STAFF table and get the STAFF unique ID. Also, by removing context of the "Role", I also made up a column for the StaffUserName. This will eliminate the duplicate records because it rolls-up the totals per PERSON, not per the persons ROLE.
select
S.UniqueStaffID,
COUNT(*) NumRecs
into
#tmpAssigned
from
tblTPRequestDetail ReqDt
JOIN [tblstaff] S
ON ReqDt.AssignedToID = S.AD_Id
INNER JOIN tblTPRequest Req
ON ReqDt.RequestID = Req.Id
AND Req.TypeOfRequest = 1
-- apply the date filters directly to where you are getting queries
AND Req.crtddate >= '2017-04-01'
and Req.crtddate <= '2017-05-30'
WHERE
-- and only getting same criteria
ReqDt.StatusCode IN ( 'COMP', 'PCR' )
OR ( ReqDt.StatusCode = 'AWIP'
and ReqDt.SubStatusCode IN ( 'BENRE', 'BENSF', 'DRR', 'DRSO', 'RPSOFF' )
)
GROUP BY
S.UniqueStaffID
select
S.UniqueStaffID,
COUNT(*) NumRecs
into
#tmpReviewed
from
tblTPRequestDetail ReqDt
JOIN [tblstaff] S
ON ReqDt.AssignedToID = S.AD_Id
INNER JOIN tblTPRequest Req
ON ReqDt.RequestID = Req.Id
AND Req.TypeOfRequest = 1
-- apply the date filters directly to where you are getting queries
AND Req.crtddate >= '2017-04-01'
and Req.crtddate <= '2017-05-30'
WHERE
-- and only getting same criteria
ReqDt.StatusCode IN ( 'COMP', 'PCR' )
OR ( ReqDt.StatusCode = 'AWIP'
and ReqDt.SubStatusCode IN ( 'BENSF', 'DRSO', 'RPSOFF', 'RPC', 'TPRPC' )
)
GROUP BY
S.UniqueStaffID
select
S.UniqueStaffID,
COUNT(*) NumRecs
into
#tmpSigned
from
tblTPRequestDetail ReqDt
JOIN [tblstaff] S
ON ReqDt.AssignedToID = S.AD_Id
INNER JOIN tblTPRequest Req
ON ReqDt.RequestID = Req.Id
AND Req.TypeOfRequest = 1
-- apply the date filters directly to where you are getting queries
AND Req.crtddate >= '2017-04-01'
and Req.crtddate <= '2017-05-30'
WHERE
-- and only getting same criteria
ReqDt.StatusCode IN ( 'COMP', 'PCR' )
OR ( ReqDt.StatusCode = 'AWIP'
and ReqDt.SubStatusCode IN ( 'RPSOFF', 'SCPC', 'TPSCPC' )
)
GROUP BY
S.UniqueStaffID
SELECT
S.StaffUserName,
REPLACE(S.Designation, 'IND ', '') as Designation,
S.Office as Location,
coalesce( Assigned.NumRecs, 0 ) [Indian Benchmarking Assigned],
coalesce( Reviewed.NumRecs, 0 ) [Indian Benchmarking Reviewed],
coalesce( Signed.NumRecs, 0 ) [Indian Benchmarking Signed],
coalesce( AllAssigned.NumRecs, 0 ) [All Indian Benchmarking Assigned],
coalesce( AllReviewed.NumRecs, 0 ) [All Indian Benchmarking Reviewed],
coalesce( AllSigned.NumRecs, 0 ) [All Indian Benchmarking Signed],
coalesce( AllAssigned.NumRecs, 0 )
+ coalesce( AllReviewed.NumRecs, 0 )
+ coalesce( AllSigned.NumRecs, 0 ) [AllRecords]
from
JOIN [tblstaff] S
LEFT JOIN #tmpAssigned Assigned
on S.UniqueStaffID = Assigned.UniqueStaffID
JOIN
( select sum(NumRecs) NumRecs
from #tmpAssigned ) AllAssigned
on 1=1
LEFT JOIN #tmpReviewed Reviewed
on S.UniqueStaffID = Reviewed.UniqueStaffID
JOIN
( select sum(NumRecs) NumRecs
from #tmpReviewed ) AllReviewed
on 1=1
LEFT JOIN #tmpSigned Signed
on S.UniqueStaffID = Signed.UniqueStaffID
JOIN
( select sum(NumRecs) NumRecs
from #tmpSigned ) AllSigned
on 1=1
WHERE
U.Active = 1
and U.RoleId !=6
ORDER BY
S.StaffUserName,
S.Designation,
S.Office
Since the pre-query of Assigned, Reviewed and Signed are pre-aggregated to at most, one user record at a time, you don't need to use a group by at the outer level. Also, for the "TOTAL" of assigned, reviewed, signed, since those are not grouped, they will always be a single record representing the entire qualified pool of records of respective categories. Should help you get your query done and totals as you were attempting.
Update per comment feedback.
I am not using any "distinct" in my query and you must have added that. As for why, I can only assume that a single user ROLE ID may be associated with multiple Staff? If you can edit your original post and share some SAMPLE Data representing such ROLES and STAFF might help (and do a couple roles/staff to see better correlation and obviously nothing truly private/confidential). This is probably the source of multiple records.
I was under the impression that the "UserADID" was a unique user, not a user ROLE as this ID was assigned to the request detail. I can only guess the impact but am now thinking you might have something like
ApprovingRole ID = 1
Office 1 Staff person ID X
Office 2 Staff Person ID Y
Office 3 Staff Person ID Z
ReviewingRole ID = 2
Office 2 Staff person ID A
Office 2 Staff Person ID Y
Office 4 Staff person ID B
SigningRole ID = 3
Office 2 Staff Person ID Y
Office 4 Staff person ID B
In the above "sample data" above, you can see that staff person "Y" is a member of all 3 roles. Staff person "B" is member of 2 roles.
etc
So which "ID" is really handled here. Is the Approving "Role" ID 1 tied to the request detail or is it the STAFF person X, Y, Z tied to.
If the data is as I have it, and from your lack of sample data, how would I know which staff site / designation the underlying counts associated with. The totals are otherwise representative of the ROLD User ID, not the STAFF User ID. Let me know otherwise and I can edit results.
So now you state there are duplicates for the same USER, not user ROLE, In that case. we need to have each prequery temp joined all the way to the USER STAFF ID. not the ROLE ID. I have modified the queries above.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.