OData IEnumerable or IQueryable

Question

I'm still pretty new to OData v3/v4 and use the OData Connect Service in Visual Studio to get data from OData servers. Now my question is what would be the best, and fastest approach to GET data en then validate it with own collection?

For example we have a Microsoft.OData.Client.DataServiceQuery of Type Employee we want to GET all employees from the server and compare it with our local dataset for example from another CRM system as List(Of Employee)

To GET data from the OData service we have to setup the DataServiceQuery for example: a simple one to GET all Employees is DataServiceQuery.GetAllPages this returns an IEnumerable(Of Employee) on which we can call the .ToList function. After this we can do every comparison and update collection between the two Lists using LINQ and/or LAMBDA.

But now if i change the approach by using DataServiceQuery.Take(Number) then it returns an IQueryable(Of Employee) on which we can also call the .ToList Or is that unnecessary with an IQueryable ?

So my question here is what would be the best approach to GET all data from the OData v4 server and then stuff it in memory and then use LINQ and/or LAMBDA to compare stuff?

Answer 1

Both of your solutions in effect will do the same thing , the only difference is that by using the .Take() you are composing a query in this case that will batch the data from the server, you might be more familiar with this being referred to as Paging .

In the OData server logic, the difference between IEnumerable an IQueryable is significant, as an IQueryable is deferred until you iterate the query, calls like .ToList() will force this iteration.

On the client side however, even though the data type is a Linq IQueryable type, when you have finished composing the query you can only iterate it once, this is also true of the IEnumerable responses from DataServiceQuery .

But now if i change the approach by using DataServiceQuery.Take(Number) then it returns an IQueryable(Of Employee) on which we can also call the.ToList Or is that unnecessary with an IQueryable?

Specifically for DataServiceQuery , given that we can only iterate the response once, as a general pattern, we usually DO call .Tolist() to forcibly receive the response from the server, and so it is obvious that it was our intent to do so.

• It's not a question of being necessary or not, it is the deliberate intent to download the data once, and store it in a list variable that allows us to iterate over it again.

The IQueryable response allows you to build the filter, order and projection expression before sending that to the server, in OData terms this should translate to $filter , $orderby , $expand , $select , $skip and $top query options to be sent with the HTTP request. If you were going to apply any of those it would have improve performance but that doesn't look to apply here.

So in regard to your question, because you are GETTING ALL of the data, there is no performance difference between these two types of responses from DataServiceQuery . When you don't want ALL rows, then one fact comes into play:

If you have limited the number of records using Where() or ApplyQueryOption() or .Take() then because the payload will be smaller than a non-limited query it will return faster, purely by virtue of the data payload over the wire being smaller.

If you were to call .ToList() and then apply filtering or limiting, the entire collection would have to be downloaded first, before your filter could be evaluated, so you get the best performance by using the IQueryable and applying filter criteria before you execute it.

Again, for your specific task, as it involves iterating over the entire collection neither of these has any inherent benefit over the other. If the record set is large though, I would encourage you to use batching/paging to process the records in smaller sets so as not to lock up the server resources, and to NOT run out of memory trying to deserialize the entire result set in a single operation.

---

These types of data synchronisations are generally expensive processes to run either way, however by performing this operation on a client you have probably picked the least efficient mechanism.

If you can, this task should be performed on the server side, that way you can reduce one whole round trip of data comparisons

---

If the comparisons cannot be performed on the server, then perhaps you can formulate you comparison into the IQueryable expression and only return the matches, or the non-matching results, that is where the power of the IQueryable comes to play, it will pass the expression to the server to perform the comparison in the database directly.