C# compiler error CS1963: "An expression tree may not contain a dynamic operation"

Question

The following code:

var lst = new List<dynamic>();
lst.Add(new
{
    Name = "Bob"
});

lst.AsQueryable().Where(x => x.Name == null);

Will produce compiler error CS1963. I understand how I can get around so this code doesn't need fixing.

My question is: why? Expression trees by nature are dynamic anyway, and having created and maintained an IQueryable implementation there isn't any obvious reason why dynamic cannot be supported in an expression tree.

What's even weirder to me is that the page for documentation on CS1963 is just missing:

https://learn.microsoft.com/en-us/do.net/csharp/language-reference/compiler-messages/cs1963

I've also tried looking in the C# 3.0 specification , but couldn't find anything.

Btw I'm wondering this because my primary database is MongoDb, which has dynamic schemas and therefore dynamic operations being supported would be very handy as otherwise you have to build strong types for every possible schema, or use a low level API like BsonDocument .

Answer 1

Your question has two parts, first of all, you asked what CS1953 means (ie why your code doesn't compile), and the other question is on why Microsoft chose to design this feature that way. I can't speak for the language design team, but I can explain the technical difficulty and from it make some guesses about the design decisions.

The technical side

The reason you got an error is that you tried to use a feature of C# that is not supported inside of a System.Linq.Expression . LINQ expressions support a subset of C#, and there are many other features of C# that you can't use inside of a LINQ expression. An example of such feature is pattern matching. The following code produced a similar compile time error to what you have seen:

queryable.Where(i => i is string s && s.Contains("a"))

There is no special reason why pattern matching is not yet supported, as far as I know. It's probably a matter of priority between new language features.

However, supporting dynamic in this context is especially difficult. Let's look at an example.

Let's say we are using a concrete compile-time type (not dynamic ). We can write something like:

public record Person(string Name);

var bob = queryable.First(person => person.Name.StartsWith("Bob"));

If we inspect the expression tree that was passed to First , we should see it contains an InstanceMethodCallExpression to the method System.String.StartsWith(System.String) .

If we would have omitted the compile-time type information (by using dynamic ), the compiler would have no way of knowing which method we wanted to call. The only things it can deduce from the expression itself is that the method is named StartsWith and it has an overload that accepts a single System.String . That means it can't construct an InstanceMethodCallExpression with an exact method, because it doesn't have enough information.

The why

As you mentioned, this is an "implementation detail". Microsoft could have modified or added expression classes that represents dynamic behavior. In fact, there is a DynamicExpression class and you can probably construct the query you were looking for using low-level expression methods (like Expression.Dynamic(...) ).

So why (according to my guess) did they choose not to support it in C#?

To understand this, let's first explore the goal of expression trees in C#. You mentioned that

Expression trees by nature are dynamic anyway

And I completely disagree with this assertion. Expression trees are a way of communicating logic in formal C# to a method that accepts an expression. The only justification for their existence is that they provide strong compile-time checks over the logic that you are passing. Otherwise, they are not better than passing query logic as magic strings (like they do in the popular Java ORM "Hibranate" and their HQL query language )

In fact, the section about LINQ in the official C# programming guide opens with the words:

Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language. Traditionally, queries against data are expressed as simple strings without type checking at compile time or IntelliSense support...

Furthermore, because a method call on a dynamic expression is linked at runtime, a LINQ provider trying to compile this expression to a different language will have a hard time doing that. This is because the actual method that it should "call" depends on type information that it does not know. So it will have to either

a. Make some assumptions, like assume everything that we try to call StartsWith on it is a string, or:

b. Delegate the polymorphic call to the other platform (in your case, MongoDB).

I argue that both of these solutions are bad.

The first one is dangerous. If the actual value is not a string, we may cause an error or worse, do something we didn't intend to do.

The second one is bad too, because it defeats the purpose of this whole feature. Instead of defining query behavior in a common standard, we delegate most of the work to the external system, that may or may not be able to handle this (which is fine), but can also interpret this in many different ways. Without a .NET runtime, you cannot know exactly what method to call on a dynamic expression. The only provider that will truly be able to implement it, is the in-memory provider (because it has access to a .NET runtime and can check the type of the value before executing).

So what should you do instead?

You obviously can't use a language feature that is not supported, but you still need to query your MongoDB collection somehow.

First of all, I have to mention (otherwise, how could this be a proper SO answer?) that if you're trying to force dynamic typing on a C# application because of your chosen persistence technology, you probably made some design mistakes. MongoDB is an excellent database engine, partly because it lets you store objects without defining schemas in advance, but you should generally still have a schema for them. If you are using a statically typed language in your application, such as C#, you typically define the schema using this language, and the difference from SQL databases is that you don't have to keep two models of the schema in sync with things like Entity Framework Migrations. But having an explicit schema is just how things are done in statically typed languages, and I strongly believe this is better for large applications (and you probably shouldn't be scripting in C#).

BUT, if you still need to do this without defining a rigid schema, you can use BsonDocument as you mentioned and you can use it with .AsQueryable() by accessing members with the indexer that accepts a string (ie x["Name"] ). This is a bit more verbose than what you would have hoped for, but magic strings are the best we can do if we don't have a rigid schema defined in a formal language.