Azure Mobile Server SDK: When the IQueryable is evaluated?

Question

I have a backend on MS Azure built on top of Azure Mobile App Service SDK (namespace Microsoft.Azure.Mobile.Server.Tables and so on). It is running ASP.NET MVC over a SQL Server database, in C#.

I have scaffolded my Controllers and I have the method GetAllTodoItems that returns an IQueryable<TodoItem> .

When exactly is this IQueryable evaluated?

I have set up a performance load test and the average request takes 46 seconds to complete, while my visible code and the SQL query takes maximum 5ms!!

What am I missing?

EDIT ====================

Here is my GetAllTodoItems method, together with dependencies:

protected IQueryable<TModelDTO> GetAllEntities()
{
    IQueryable<TModel> allEntitiesQuery = Query();
    IQueryable<string> visibleObj = context.VisibleObjs(GetUserID(), AttType);

    IQueryable<TModel> finalQuery = from item in allEntitiesQuery
                                    join visib in visibleObj on item.Id equals visib
                                    select item;
    return finalQuery.Select(Selector).AsQueryable();
}

IQueryable<string> VisibleObjs(string userID, AttachmentType type)
{
    return (from ud in UserDesktops
            join a in Attachments on ud.DesktopId equals a.ParentDesktop
            where (ud.UserId == userID) && (a.AttachmentType == type))
            select a.Id);
}

protected Func<TModel, TModelDTO> Selector { get { return d => ToDTO(d); } }

protected override TModelDTO ToDTO(TModel input)
{
    return new TModelDTO(input);
}

public TModelDTO(TModel entity)
{
    // all basic properties copied:
    Content = entity.Content;
    Width = entity.Width;
    Color = entity.Color;
    HighResImageContent = entity.HighResImageContent;
    ImageContent = entity.ImageContent;
    MaskPath = entity.MaskPath;
    MinHeight = entity.MinHeight;
    IsComment = entity.IsComment;
    IsInkNote = entity.IsInkNote;
}

Answer 1

In this case it could be executing in several locations. Pull the GetUserID method out and put it in a variable above.

var userId = GetUserID();
IQueryable<string> visibleObj = context.VisibleObjs(userId, AttType);

That may solve your performance problem right there - perhaps it is executing that SQL separately, then joining in memory.

In addition, context.VisibleObjs - is context the same context used by Query()? If it is a different context, this won't use SQL to join. You should be getting the context in the Initialize method of the controller and storing it in a class variable there.

Also, what type is AttachmentType ? Is it an enum? Perhaps needs cast to an int explicitly? Need more info there.

When exactly is this IQueryable evaluated?

The point at which we want the SQL to run is when it is iterated. In the code above it actually should not run within this method if written correctly. When it is iterated , all expressions before finalQuery.Select(Selector) should be translated into SQL. The Selector method obviously can't be run on the database, so at that time it requires to run the SQL as the query unwinds itself.

The query will unwind itself during serialization.

What does this mean? Well you've handed back to the API an IQueryable object made up of an Expression Tree . The Table Service framework may add some filters or sorts as requested by the web client ( see supported query operators ). After doing that the web api framework (which called into your controller) will enumerable the IQueryable triggering execution as it writes out JSON(?).

We need to know what SQL is actually running. That's key to working with Linq to SQL / EF.

When faced with troubleshooting Linq to SQL I often put a logger on the context's database. context.Database.Log = Console.Write is the quick solution I use. With a TableController, you would want context.Database.Log = a => this.Configuration.Services.GetTraceWriter().Info(a); in the Initialize method of your controller - where the context is initialized.

Then simply take a look at the log.

I mocked up tables, schema, etc into a TableController and ran this myself, then went through the output with the Logging hooked up, so lets take a look at what's happening :

iisexpress.exe Information: 0 : Request, Method=GET, Url=http://localhost:51543/tables/TodoItem?ZUMO-API-VERSION=2.0.0, Message='http://localhost:51543/tables/TodoItem?ZUMO-API-VERSION=2.0.0'
iisexpress.exe Information: 0 : Message='TodoItem', Operation=DefaultHttpControllerSelector.SelectController
iisexpress.exe Information: 0 : Message='maqsService.Controllers.TodoItemController', Operation=DefaultHttpControllerActivator.Create
iisexpress.exe Information: 0 : Message='maqsService.Controllers.TodoItemController', Operation=HttpControllerDescriptor.CreateController
iisexpress.exe Information: 0 : Message='Selected action 'GetAllTodoItems()'', Operation=ApiControllerActionSelector.SelectAction
iisexpress.exe Information: 0 : Operation=HttpActionBinding.ExecuteBindingAsync
iisexpress.exe Information: 0 : Operation=TableQueryFilter.OnActionExecutingAsync
iisexpress.exe Information: 0 : Operation=EnableQueryAttribute.OnActionExecutingAsync
iisexpress.exe Information: 0 : Operation=TableControllerConfigAttribute.OnActionExecutingAsync
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131799606929250512): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Numerics\v4.0_4.0.0.0__b77a5c561934e089\System.Numerics.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131799606929250512): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_32\System.Data.OracleClient\v4.0_4.0.0.0__b77a5c561934e089\System.Data.OracleClient.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.

Next is the line in the log indicating that it got back the IQueryable:

iisexpress.exe Information: 0 : Message='Action returned 'System.Linq.Enumerable+WhereSelectEnumerableIterator`2[maqsService.DataObjects.TodoItem,maqsService.Controllers.TodoItemDTO]'', Operation=ReflectedHttpActionDescriptor.ExecuteAsync

Note, no SQL has been executed. It now identifies that it wants to serialize that in Json:

iisexpress.exe Information: 0 : Message='Will use same 'JsonMediaTypeFormatter' formatter', Operation=JsonMediaTypeFormatter.GetPerRequestFormatterInstance
iisexpress.exe Information: 0 : Message='Selected formatter='JsonMediaTypeFormatter', content-type='application/json; charset=utf-8'', Operation=DefaultContentNegotiator.Negotiate
iisexpress.exe Information: 0 : Operation=ApiControllerActionInvoker.InvokeActionAsync, Status=200 (OK)
iisexpress.exe Information: 0 : Operation=TableControllerConfigAttribute.OnActionExecutedAsync, Status=200 (OK)

Now the JsonSerializer is going to serialize the IQueryable, and to do so it needs to enumerate it.

iisexpress.exe Information: 0 : Message='Opened connection at 8/28/2018 4:11:48 PM -04:00
'
'iisexpress.exe' (CLR v4.0.30319: /LM/W3SVC/2/ROOT-1-131799606929250512): Loaded 'EntityFrameworkDynamicProxies-maqsService'. 
iisexpress.exe Information: 0 : Message='SELECT 
    [Extent1].[Id] AS [Id], 
    [Extent1].[Text] AS [Text], 
    [Extent1].[Complete] AS [Complete], 
    [Extent1].[AttachmentId] AS [AttachmentId], 
    [Extent1].[Version] AS [Version], 
    [Extent1].[CreatedAt] AS [CreatedAt], 
    [Extent1].[UpdatedAt] AS [UpdatedAt], 
    [Extent1].[Deleted] AS [Deleted]
    FROM  [dbo].[TodoItems] AS [Extent1]
    INNER JOIN  (SELECT [Extent2].[UserId] AS [UserId], [Extent3].[Id] AS [Id1], [Extent3].[AttachmentType] AS [AttachmentType]
        FROM  [dbo].[UserDesktops] AS [Extent2]
        INNER JOIN [dbo].[Attachments] AS [Extent3] ON [Extent2].[DesktopId] = [Extent3].[ParentDesktop] ) AS [Join1] ON [Extent1].[Id] = [Join1].[Id1]
    WHERE (([Join1].[UserId] = @p__linq__0) OR (([Join1].[UserId] IS NULL) AND (@p__linq__0 IS NULL))) AND ([Join1].[AttachmentType] = @p__linq__1)'
iisexpress.exe Information: 0 : Message='
'
iisexpress.exe Information: 0 : Message='-- p__linq__0: 'dana' (Type = String, Size = 4000)
'
iisexpress.exe Information: 0 : Message='-- p__linq__1: '1' (Type = Int32, IsNullable = false)
'
iisexpress.exe Information: 0 : Message='-- Executing at 8/28/2018 4:11:48 PM -04:00
'
iisexpress.exe Information: 0 : Message='-- Completed in 7 ms with result: SqlDataReader
'
iisexpress.exe Information: 0 : Message='
'
iisexpress.exe Information: 0 : Message='Closed connection at 8/28/2018 4:11:48 PM -04:00
'

SQL complete.

iisexpress.exe Information: 0 : Operation=EnableQueryAttribute.OnActionExecutedAsync, Status=200 (OK)
iisexpress.exe Information: 0 : Operation=TableQueryFilter.OnActionExecutedAsync, Status=200 (OK)
iisexpress.exe Information: 0 : Operation=TodoItemController.ExecuteAsync, Status=200 (OK)
iisexpress.exe Information: 0 : Response, Status=200 (OK), Method=GET, Url=http://localhost:51543/tables/TodoItem?ZUMO-API-VERSION=2.0.0, Message='Content-type='application/json; charset=utf-8', content-length=unknown'
iisexpress.exe Information: 0 : Operation=JsonMediaTypeFormatter.WriteToStreamAsync
iisexpress.exe Information: 0 : Operation=TodoItemController.Dispose

I'm not sure what the performance problems are. While my table has no data I see that it is properly rolling everything up into one SQL. Something that could be different? The type for AttachmentType. I used an int.

What else?

If you want to own the stack and truly understand what's going on under the sheets, There is a series of OData framework filters put on these that are up the call stack also.

This is the stack during the call to the TModelDTO (TodoItemDTO in my example) constructor, which is called during iteration over the SQL result. Note that the stack doesn't have our controller method in in. We've long since handed back the IQueryable . This is back in the framework code where it's actually using that IQueryable, which I'm intercepting because our Select method calls into the DTO to transform it.

maqsService.dll!maqsService.Controllers.TodoItemDTO.TodoItemDTO(maqsService.DataObjects.TodoItem entity) Line 89    C#
    maqsService.dll!maqsService.Controllers.TodoItemController.ToDTO(maqsService.DataObjects.TodoItem input) Line 55    C#
    maqsService.dll!maqsService.Controllers.TodoItemController.get_Selector.AnonymousMethod__6_0(maqsService.DataObjects.TodoItem d) Line 51    C#
>   System.Core.dll!System.Linq.Enumerable.WhereSelectEnumerableIterator<maqsService.DataObjects.TodoItem, maqsService.Controllers.TodoItemDTO>.MoveNext()  Unknown
    System.Core.dll!System.Linq.Buffer<maqsService.Controllers.TodoItemDTO>.Buffer(System.Collections.Generic.IEnumerable<maqsService.Controllers.TodoItemDTO> source)  Unknown
    System.Core.dll!System.Linq.OrderedEnumerable<maqsService.Controllers.TodoItemDTO>.GetEnumerator()  Unknown
    System.Core.dll!System.Linq.Enumerable.TakeIterator<maqsService.Controllers.TodoItemDTO>(System.Collections.Generic.IEnumerable<maqsService.Controllers.TodoItemDTO> source, int count) Unknown
    mscorlib.dll!System.Collections.Generic.List<maqsService.Controllers.TodoItemDTO>.List(System.Collections.Generic.IEnumerable<maqsService.Controllers.TodoItemDTO> collection) Line 99  C#

It appears that here, OData evaluating the $top query parameter for a page number does actually then put the results into a list.

System.Web.Http.OData.dll!System.Web.Http.OData.Query.TruncatedCollection<maqsService.Controllers.TodoItemDTO>.TruncatedCollection(System.Linq.IQueryable<maqsService.Controllers.TodoItemDTO> source, int pageSize)    Unknown
        System.Web.Http.OData.dll!System.Web.Http.OData.Query.ODataQueryOptions.LimitResults<maqsService.Controllers.TodoItemDTO>(System.Linq.IQueryable<maqsService.Controllers.TodoItemDTO> queryable, int limit, out bool resultsLimited)    Unknown

I believe the native transition here is SQL related. I can't find exactly whats up in the decompiled source though.

    [Native to Managed Transition]  
    [Managed to Native Transition]  
System.Web.Http.OData.dll!System.Web.Http.OData.Query.ODataQueryOptions.LimitResults(System.Linq.IQueryable queryable, int limit, out bool resultsLimited)  Unknown
        System.Web.Http.OData.dll!System.Web.Http.OData.Query.ODataQueryOptions.ApplyTo(System.Linq.IQueryable query, System.Web.Http.OData.Query.ODataQuerySettings querySettings) Unknown
        System.Web.Http.OData.dll!System.Web.Http.OData.EnableQueryAttribute.ApplyQuery(System.Linq.IQueryable queryable, System.Web.Http.OData.Query.ODataQueryOptions queryOptions)   Unknown
        System.Web.Http.OData.dll!System.Web.Http.OData.EnableQueryAttribute.ExecuteQuery(object response, System.Net.Http.HttpRequestMessage request, System.Web.Http.Controllers.HttpActionDescriptor actionDescriptor)   Unknown
        System.Web.Http.OData.dll!System.Web.Http.OData.EnableQueryAttribute.OnActionExecuted(System.Web.Http.Filters.HttpActionExecutedContext actionExecutedContext)  Unknown

To dig even deeper , how does the framework know what to do with the IQueryable anyhow?! Well when it iterates it, the IQueryable uses its IQueryProvider to parse out the contained Expression (note this is not compiled code, this is a tree of method calls and operators which you added using your joins and where clauses). It transforms that tree into SQL (in this case) as best it can. When it hits something it can't translate it throws up an error or finds a way to work around.

Gaining a deep understanding of IQueryProvider is a fairly complex computer science task. You could get started here with a walkthrough of creating a Query Provider. Once upon a time I wrote a query provider to transform Linq expressions into Ektron CMS API calls, and you could take a look at that here . I wrote a pretty good summary with links to key areas.

I hope that helped. Not sure what else I could have gone deeper into, and thank you for teaching me something new today. I had no clue what this mobile table API is (still not clear on the point of it)

Answer 2

Linq is lazy, which means that it is executed actually when you try to access the enumeration like in a foreach loop or some extension methods like .ToList() or .ToArray(). When you define the linq query in your method, than it is simple a "preparation" what should be done, when accessing the result. The preparation of the query takes just a little moment in contrast of the execution. This is the reason why you see that your own code runs in a few milliseconds. It's only a preparation. Finally when you access the result, the query is executed actually, eg when asp.net serializes your data to build the response of a request.

In your case you try to build a case insensitive filter

where (ud.UserId.Equals(userID, StringComparison.InvariantCultureIgnoreCase)

in VisibleObjs() method. The Equals(userID, StringComparison.InvariantCultureIgnoreCase) call seems to enforce EF to query/return all data from the table before processing the filter when executed. The evaluation of your filter is executed on the client side instead of using a case insensitive search on the sqlserver. One possible solution can be to mark your sqlserver column UserDesktops.UserId in the database with a collation "SQL_Latin1_General_CP1_CI_AS" where CI means CaseInsensitive. After that you should replace your filter by

where (ud.UserId == userID)

or something similar without using any .Net methods to allow EF to translate your linq filter to a plain sql comparison. In this case the case insensitive filter is processed by sqlserver directly without requesting the full table from sqlserver and filtering on client side.

Answer 3

For the action which returns IQueryable data in ApiController , the Web API will do ToList operation and then serializes the list value, finally writes the serialized the list into the response body, and the response status code is 200 (OK).

Only when we execute the "ToList" method of IQueryable , the data in database is actually taken and the "Excute" method in "IQueryProvider" is executed. (parses the expression, and then executes to get the result).

As you said that it takes 46 seconds to complete, i guess that you do some time-consuming operation with IQueryable , for example: Taking IEnumerable first and then filter the data will cause performance problems.

You can provide us with more detailed code for further research.

Hope this was helpful.