Why do I have to Force Order with these hierarchy queries/

Question

Below is an example of a query I might run where, for each category, I want the NumberOfCourses to represent not only that specific category but also any child categories under it. I think the query is fairly self explanatory.

select  c.CategoryID, courses.MarketID, count(distinct courses.CourseID) NumberOfCourses
from    Category c
        join CategoryHierarchy tch on tch.HierarchyKey like '%~' + cast(c.CategoryID as varchar) + '~%'
        join vLiveEvents courses on tch.CategoryID = courses.CategoryID
where   courses.MarketID is not null
group by c.CategoryHumanID, courses.MarketID

When I run this as is it may take almost two minutes, however if I add the hint Option (Force Order) then it only takes a few seconds to run. So my question is am I doing something wrong that's causing SQL to create a bad plan or is the SQL engine actually just not good at optimizing hierarchy joins like this?

I tried including the sql plan, but it's way too long and SO won't let me have that many characters. I'm happy to share it though if anyone can tell me how to do so.

EDIT: I guess probably not every knows how these kinds of hierarchies work. They hierarchy key will look something like ~1234~5678~9123~ where 1234 is the parent of 5678 which is the parent of 9123. By doing a like comparison on a CategoryID I can include all child categories in the results.

Answer 1

Starting from SQL Server 2016+, Query Store feature was introduced to monitor performance. It provides insight into query plan choice and performance.

It also provides an option to force plan.

It's not a complete replacement of trace or extended events, but as it's evolving from version to version, we might get a fully functional query store in future releases from SQL Server. The primary flow of Query Store

1. SQL Server existing components interact with query store by utilising Query Store Manager.
2. Query Store Manager determines which Store should be used and then passes execution to that store (Plan or Runtime Stats or Query Wait Stats)
   • Plan Store - Persisting the execution plan information
   • Runtime Stats Store - Persisting the execution statistics information
   • Query Wait Stats Store - Persisting wait statistics information.
3. Plan, Runtime Stats and Wait store uses Query Store as an extension to SQL Server.

1. Enabling the Query Store : Query Store works at the database level on the server.

   • Query Store is not active for new databases by default.
   • You cannot enable the query store for the master or tempdb database.
   • Available DMV

     sys.database_query_store_options (Transact-SQL)

2. Collect Information in the Query Store : We collect all the available information from the three stores using Query Store DMV (Data Management Views).

   • Query Plan Store: Persisting the execution plan information and it is accountable for capturing all information that is related to query compilation.

     sys.query_store_query (Transact-SQL) sys.query_store_plan (Transact-SQL) sys.query_store_query_text (Transact-SQL)

   • Runtime Stats Store: Persisting the execution statistics information and it is probably the most frequently updated store. These statistics represent query execution data.

     sys.query_store_runtime_stats (Transact-SQL)

   • Query Wait Stats Store: Persisting and capturing wait statistics information.

     sys.query_store_wait_stats (Transact-SQL)

NOTE: Query Wait Stats Store is available only in SQL Server 2017+