Entity Framework Core 3.1 stored procedure optional parameters not working

Question

[SOLVED] (see the end)

I've spent most of the day looking for a solution that works with no luck.

• EF Core 3.1
• Visual Studio 2019
• SQL Server 2017
• .Net Core 3.1

I have not been able to find a solution that works to call a stored procedure from.Net Core 3.1 with optional parameters. I've read dozens of articles from SO to blogs, but nothing works.

Here's my C# code for the method to call the stored procedure with all the attempts at making this work:

public List<BudgetWorkflowStatusByDepartment> GetBudgetWorkflowStatusByDepartment(int? companyId, int? departmentId, int? locationId, int? sublocationId)
{
        //var compId = new SqlParameter("@CompanyId", companyId);
        //compId.Value = (object)companyId ?? SqlInt32.Null;

        //var deptId = new SqlParameter("@DepartmentId", departmentId);
        //deptId.Value = (object)departmentId ?? SqlInt32.Null;

        //var locId = new SqlParameter("@LocationId", locationId);
        //locId.Value = (object)locationId ?? SqlInt32.Null;

        //var subId = new SqlParameter("@SubLocationId", sublocationId);
        //subId.Value = (object)sublocationId ?? SqlInt32.Null;

        var compId = new SqlParameter("@CompanyID", companyId);
        compId.Value = (object)companyId ?? DBNull.Value;

        var deptId = new SqlParameter("@DepartmentID", departmentId);
        deptId.Value = (object)departmentId ?? DBNull.Value;

        var locId = new SqlParameter("@LocationID", locationId);
        locId.Value = (object)locationId ?? DBNull.Value;

        var subId = new SqlParameter("@SubLocationID", sublocationId);
        subId.Value = (object)sublocationId ?? DBNull.Value;

        //var result = context.BudgetWorkflowStatusByDepartment
        //    .FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment @CompanyID, @DepartmentID, @LocationID, @SubLocationID", compId, deptId, locId, subId).ToList();

        //var result = context.BudgetWorkflowStatusByDepartment
        //    .FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment {0}, {1}, {2}, {3}", companyId, departmentId, locationId, companyId).ToList();

        // Error Must declare the scalar variable "@CompanyID".
        //var result = context.BudgetWorkflowStatusByDepartment
        //    .FromSqlRaw($"EXEC mis.BudgetWorkflowStatusByDepartment @CompanyID={compId}, @DepartmentID={deptId}, @LocationID={locId}, @SubLocationID={subId}").ToList();

        // Error Must declare the scalar variable "@CompanyID".
        var result = context.BudgetWorkflowStatusByDepartment
            .FromSqlRaw($"EXEC mis.BudgetWorkflowStatusByDepartment {compId}, {deptId}, {locId}, {subId}").ToList();

        //var result = context.BudgetWorkflowStatusByDepartment
        //        .FromSqlInterpolated($"EXEC mis.BudgetWorkflowStatusByDepartment @CompanyID={compId}, @DepartmentID={deptId}, @LocationID={locId}, @SubLocationID={subId}").ToList();

        return result;
}

None of these attempts work. The errors range from:

Data is Null. This method or property cannot be called on Null values.

... to...

Must declare the scalar variable "@compId".

... to some other errors when I just tried something that was most likely just incorrect in a desperate attempt to get it to work.

This is the signature of the stored procedure itself showing that all parameters are nullable.

ALTER PROCEDURE [mis].[BudgetWorkflowStatusByDepartment] 
    (@CompanyID int = null,
     @DepartmentID int = null,
     @LocationID int = null,
     @SubLocationID int = null)

The frustrating part is that this stored procedure works, in SQL Server, in LinqPad with or without parameters. And it works in my code only with ALL the parameters, but if any are null, it throws an error.

Any insight on how I can get the function to work with optional parameters, is appreciated.

Edited: 8/24/2020

Based on the new suggestions, I tried them and for some reason they still don't work.

I've tested the sproc directly in SSMS with all four parameters, and it works. Then I removed each of the last params one by one and tested it as each was removed and it works returning correct data.

When I do the same test using the Api and the service call to this method, the four params work, 3 params work, but 2 params, 1 param and no params, don't work. I don't understand why not.

Here are my new attempts:

            /// New attempts as of: 8/24/2020
            // Error Data is Null. This method or property cannot be called on Null values.
            //var compId = new SqlParameter("@CompanyID", companyId);
            //compId.Value = (object)companyId ?? DBNull.Value;

            //var deptId = new SqlParameter("@DepartmentID", departmentId);
            //deptId.Value = (object)departmentId ?? DBNull.Value;

            //var locId = new SqlParameter("@LocationID", locationId);
            //locId.Value = (object)locationId ?? DBNull.Value;

            //var subId = new SqlParameter("@SubLocationID", sublocationId);
            //subId.Value = (object)sublocationId ?? DBNull.Value;

            //var result = context.BudgetWorkflowStatusByDepartment
            //        .FromSqlInterpolated($"EXEC mis.BudgetWorkflowStatusByDepartment {compId}, {deptId}, {locId}, {subId}").ToList();

            // Error Data is Null. This method or property cannot be called on Null values.
            
            //var result = context.BudgetWorkflowStatusByDepartment
            //        .FromSqlInterpolated($"EXEC mis.BudgetWorkflowStatusByDepartment {companyId}, {departmentId}, {locationId}, {sublocationId}").ToList();


            //var compId = new SqlParameter("@CompanyID", companyId) { IsNullable = true };
            //compId.Value = (object)companyId ?? DBNull.Value;

            //var deptId = new SqlParameter("@DepartmentID", departmentId) { IsNullable = true };
            //deptId.Value = (object)departmentId ?? DBNull.Value;

            //var locId = new SqlParameter("@LocationID", locationId) { IsNullable = true };
            //locId.Value = (object)locationId ?? DBNull.Value;

            //var subId = new SqlParameter("@SubLocationID", sublocationId) { IsNullable = true };
            //subId.Value = (object)sublocationId ?? DBNull.Value;

            //// Error Data is Null. This method or property cannot be called on Null values.
            //var result = context.BudgetWorkflowStatusByDepartment
            //                    .FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment @CompanyID = @CompanyID, @DepartmentID = @DepartmentID, @LocationID = @LocationID, @SubLocationID = @SubLocationID", compId, deptId, locId, subId)
            //                    .ToList();

            var compId = new SqlParameter("@CompanyID", companyId) { IsNullable = true };
            compId.Value = (object)companyId ?? DBNull.Value;

            var deptId = new SqlParameter("@DepartmentID", departmentId) { IsNullable = true };
            deptId.Value = (object)departmentId ?? DBNull.Value;

            var locId = new SqlParameter("@LocationID", locationId) { IsNullable = true };
            locId.Value = (object)locationId ?? DBNull.Value;

            var subId = new SqlParameter("@SubLocationID", sublocationId) { IsNullable = true };
            subId.Value = (object)sublocationId ?? DBNull.Value;

            // Error Data is Null. This method or property cannot be called on Null values.
            var result = context.BudgetWorkflowStatusByDepartment
                                .FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment @CompanyID, @DepartmentID, @LocationID, @SubLocationID", compId, deptId, locId, subId)
                                .ToList();

I thought I should include my DbContext property settings for this sproc as additional info. This same pattern is used for two other sprocs and they work but they don't require optional parameters.

        public DbSet<BudgetWorkflowStatusByDepartment> BudgetWorkflowStatusByDepartment { get; set; }

And in the OnModelCreating() method I set the HasNoKey() method.

            modelBuilder.Entity<BudgetWorkflowStatusByDepartment>().HasNoKey();

I appreciate everyone's suggestions.

My work around on this will be to tell the DBA to change the sproc to remove the parameters all together, and I'll just do a .Where() clause on the result.

Luckily for this particular feature it won't be returning more than 200 rows, but it will return many columns, but this still should not be taxing.

Solved reason

It was my fault. I had some int properties that needed to be nullable ints. Once I did that, it worked, and it seems to work with most solutions.

It's important to pay attention to those properties that could be null and make them nullable, so you don't fall prey to my issue, that took an entire day to solve.

---

I thought that since I was going through all these attempts at something I hadn't done before (working with optional params with.Net Core 3.1 stored procedures), that I'd document my results with the various options in handling this.

These are some options on what works and what doesn't when executing a stored procedure from.Net Core 3.1.

For these examples, this will be the signature of the.Net Core method.

public List<BudgetWorkflowStatusByDepartment> GetBudgetWorkflowStatusByDepartment
(int? companyId, int? departmentId, int? locationId, int? sublocationId)
{
...
}

All of the parameters are optional. The following examples would be the code you could use in this method.

This is the signature for the stored procedure.

ALTER procedure mis.BudgetWorkflowStatusByDepartment_Filtered (
      @CompanyID int = null
    , @DepartmentID int = null
    , @LocationID int = null
    , @SubLocationID int = null
)

Working Examples

The following examples demonstrate how to use either the FromSqlRaw or FromSqlInterpolated methods to execute the stored procedure.

FromSqlRaw Methods

This example simply creates SqlParameter objects and sets the values to the incoming value, or DbNull . This is important as null won't work in this case. (See Non Working Examples below)

// Works
var compId = new SqlParameter("@CompanyID", companyId);
compId.Value = (object)companyId ?? DBNull.Value;

var deptId = new SqlParameter("@DepartmentID", departmentId);
deptId.Value = (object)departmentId ?? DBNull.Value;

var locId = new SqlParameter("@LocationID", locationId);
locId.Value = (object)locationId ?? DBNull.Value;

var subId = new SqlParameter("@SubLocationID", sublocationId);
subId.Value = (object)sublocationId ?? DBNull.Value;

var result = context.BudgetWorkflowStatusByDepartment
.FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment_Filtered @CompanyID, @DepartmentID, @LocationID, @SubLocationID", compId, deptId, locId, subId).ToList();

---

The following works by including the IsNullable=true property of the SqlParameter object.

// Works
var compId = new SqlParameter("@CompanyID", companyId) { IsNullable = true };
compId.Value = (object)companyId ?? DBNull.Value;

var deptId = new SqlParameter("@DepartmentID", departmentId) { IsNullable = true };
deptId.Value = (object)departmentId ?? DBNull.Value;

var locId = new SqlParameter("@LocationID", locationId) { IsNullable = true };
locId.Value = (object)locationId ?? DBNull.Value;

var subId = new SqlParameter("@SubLocationID", sublocationId) { IsNullable = true };
subId.Value = (object)sublocationId ?? DBNull.Value;

var result = context.BudgetWorkflowStatusByDepartment
        .FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment_Filtered @CompanyID = @CompanyID, @DepartmentID = @DepartmentID, @LocationID = @LocationID, @SubLocationID = @SubLocationID", compId, deptId, locId, subId)
        .ToList();

---

The following is similar to above but a slightly short-hand syntax method.

// Works
var compId = new SqlParameter("@CompanyID", companyId) { IsNullable = true };
compId.Value = (object)companyId ?? DBNull.Value;

var deptId = new SqlParameter("@DepartmentID", departmentId) { IsNullable = true };
deptId.Value = (object)departmentId ?? DBNull.Value;

var locId = new SqlParameter("@LocationID", locationId) { IsNullable = true };
locId.Value = (object)locationId ?? DBNull.Value;

var subId = new SqlParameter("@SubLocationID", sublocationId) { IsNullable = true };
subId.Value = (object)sublocationId ?? DBNull.Value;

// Works
var result = context.BudgetWorkflowStatusByDepartment
        .FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment_Filtered @CompanyID, @DepartmentID, @LocationID, @SubLocationID", compId, deptId, locId, subId)
        .ToList();

---

FromSqlInterpolated

The following works because it uses the .FromSqlInterpolated() method and then the placeholder syntax will work.

// Works
var compId = new SqlParameter("@CompanyID", companyId);
compId.Value = (object)companyId ?? DBNull.Value;

var deptId = new SqlParameter("@DepartmentID", departmentId);
deptId.Value = (object)departmentId ?? DBNull.Value;

var locId = new SqlParameter("@LocationID", locationId);
locId.Value = (object)locationId ?? DBNull.Value;

var subId = new SqlParameter("@SubLocationID", sublocationId);
subId.Value = (object)sublocationId ?? DBNull.Value;

var result = context.BudgetWorkflowStatusByDepartment
    .FromSqlInterpolated($"EXEC mis.BudgetWorkflowStatusByDepartment_Filtered @CompanyID={compId}, @DepartmentID={deptId}, @LocationID={locId}, @SubLocationID={subId}").ToList();

---

The following is similar to above, but it uses a slightly short-hand syntax.

// Works
var compId = new SqlParameter("@CompanyID", companyId);
compId.Value = (object)companyId ?? DBNull.Value;

var deptId = new SqlParameter("@DepartmentID", departmentId);
deptId.Value = (object)departmentId ?? DBNull.Value;

var locId = new SqlParameter("@LocationID", locationId);
locId.Value = (object)locationId ?? DBNull.Value;

var subId = new SqlParameter("@SubLocationID", sublocationId);
subId.Value = (object)sublocationId ?? DBNull.Value;

var result = context.BudgetWorkflowStatusByDepartment
    .FromSqlInterpolated($"EXEC mis.BudgetWorkflowStatusByDepartment_Filtered {compId}, {deptId}, {locId}, {subId}").ToList();

---

The following is the ultimate in short-hand syntax. With the interpolated method, you can directly use the incoming variables and not have to create the SqlParameter objects.

// Works
var result = context.BudgetWorkflowStatusByDepartment
    .FromSqlInterpolated($"EXEC mis.BudgetWorkflowStatusByDepartment_Filtered {companyId}, {departmentId}, {locationId}, {sublocationId}").ToList();

---

Non Working Examples

The following code returns an empty array, which is not correct. This parameterization of the code can only be used this way with the "interpolated" Sql method.

// Returns empty array
var result = context.BudgetWorkflowStatusByDepartment
.FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment_Filtered {0}, {1}, {2}, {3}", companyId, departmentId, locationId, companyId).ToList();

Result - an empty array, which is incorrect:

[]

---

The following method is incorrect in the syntax of the FromSqlRaw method. The parameters in the string want to use interpolation, but this is the wrong method for that syntax.

// Error Must declare the scalar variable "@CompanyID".
var compId = new SqlParameter("@CompanyID", companyId);
compId.Value = (object)companyId ?? DBNull.Value;

var deptId = new SqlParameter("@DepartmentID", departmentId);
deptId.Value = (object)departmentId ?? DBNull.Value;

var locId = new SqlParameter("@LocationID", locationId);
locId.Value = (object)locationId ?? DBNull.Value;

var subId = new SqlParameter("@SubLocationID", sublocationId);
subId.Value = (object)sublocationId ?? DBNull.Value;

var result = context.BudgetWorkflowStatusByDepartment
.FromSqlRaw($"EXEC mis.BudgetWorkflowStatusByDepartment_Filtered @CompanyID={compId}, @DepartmentID={deptId}, @LocationID={locId}, @SubLocationID={subId}").ToList();

Error:

Must declare the scalar variable "@CompanyID".

---

The following is also incorrect. The parameters in the string cannot be populated in this manner.

// Error Must declare the scalar variable "@CompanyID".
var compId = new SqlParameter("@CompanyID", companyId);
compId.Value = (object)companyId ?? DBNull.Value;

var deptId = new SqlParameter("@DepartmentID", departmentId);
deptId.Value = (object)departmentId ?? DBNull.Value;

var locId = new SqlParameter("@LocationID", locationId);
locId.Value = (object)locationId ?? DBNull.Value;

var subId = new SqlParameter("@SubLocationID", sublocationId);
subId.Value = (object)sublocationId ?? DBNull.Value;

var result = context.BudgetWorkflowStatusByDepartment
.FromSqlRaw($"EXEC mis.BudgetWorkflowStatusByDepartment_Filtered {compId}, {deptId}, {locId}, {subId}").ToList();

Error:

Must declare the scalar variable "@CompanyID".

---

And lastly, this will also not work because the ?? operator is setting null values to null instead of DbNull.Value .

var compId = new SqlParameter("@CompanyID", companyId);
compId.Value = (object)companyId ?? null;

var deptId = new SqlParameter("@DepartmentID", departmentId);
deptId.Value = (object)departmentId ?? null;

var locId = new SqlParameter("@LocationID", locationId);
locId.Value = (object)locationId ?? null;

var subId = new SqlParameter("@SubLocationID", sublocationId);
subId.Value = (object)sublocationId ?? null;

var result = context.BudgetWorkflowStatusByDepartment
.FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment_Filtered @CompanyID, @DepartmentID, @LocationID, @SubLocationID", compId, deptId, locId, subId).ToList();

Error:

The parameterized query '(@CompanyID nvarchar(4000),@DepartmentID nvarchar(4000),@Locatio' expects the parameter '@CompanyID', which was not supplied.

Answer 1

I just re-created a similar setup (same versions of everything you have), and the following one-liner works for me:

public List<BudgetWorkflowStatusByDepartment> GetBudgetWorkflowStatusByDepartment(int? companyId, int? departmentId, int? locationId, int? sublocationId)
     => context.BudgetWorkflowStatusByDepartment
               .FromSqlInterpolated($"EXEC mis.BudgetWorkflowStatusByDepartment {companyId}, {departmentId}, {locationId}, {subLocationId}").ToList();

Note, since I'm using the .FromSqlInterpolated() call, there is no need to create SqlParameter s - the int? can be passed in directly, as this is a safe call, as per docs , and the null values are properly handled.

I see that you tried this call as well, but it looks like you're passing SqlParameter s to it, not the data itself - try my version?

Upon further testing, I got your version to work, with slight modifications, and by setting SqlParameter.IsNullable property to true :

public List<BudgetWorkflowStatusByDepartment> GetBudgetWorkflowStatusByDepartment(int? companyId, int? departmentId, int? locationId, int? sublocationId)
{
    var compId = new SqlParameter("@CompanyID", companyId) { IsNullable = true };
    compId.Value = (object)companyId ?? DBNull.Value;

    var deptId = new SqlParameter("@DepartmentID", departmentId) { IsNullable = true };
    deptId.Value = (object)departmentId ?? DBNull.Value;

    var locId = new SqlParameter("@LocationID", locationId) { IsNullable = true };
    locId.Value = (object)locationId ?? DBNull.Value;

    var subId = new SqlParameter("@SubLocationID", sublocationId) { IsNullable = true };
    subId.Value = (object)sublocationId ?? DBNull.Value;


    var result = context.BudgetWorkflowStatusByDepartment
                        .FromSqlRaw("EXEC mis.BudgetWorkflowStatusByDepartment @CompanyID = @CompanyID, @DepartmentID = @DepartmentID, @LocationID = @LocationID, @SubLocationID = @SubLocationID", compId, deptId, locId, subId)
                        .ToList();

    return result;
}

Take your pick - I personally like the one-liner:)

Answer 2

The only way I have found to get named parameters (and thus optional parameters) working with do.netcore is to use the following method:

_context.Database.ExecuteSqlRaw("EXEC mysproc @param1={0}, @param2={1}"
        ,param1
        ,param2);

... where param1 and param2 are of type Microsoft.Data.SqlClient.SqlParameter (this is important, as I have found the hard way that type System.Data.SqlClient.SqlParamater will not generate compile errors but results in confusing exceptions at run time).

While this works consistently, even with output parameters (with the addition of "out" after the parameter placeholder), it is a very inflexible way of creating a statement when your code might provide different outcomes with different sets of parameters. One of my use cases has 23 parameters that I might supply, only three of them are required. Hardcoding the ExecuteSqlRaw statement is not an option.

I have since come up with a useful method which I use from my Controllers, Pages, etc. where the DbContext _context is injected:

private void ExecuteStoredProcedure(
        string StoredProcedureName
        ,List<SqlParameter> parameters)
{           
    StringBuilder stringBuilder = null;
    int paramNum = 0;

    // Build the T-SQL statement to enable named paramaters
    foreach (SqlParameter param in parameters)
    {
        string direction = param.Direction == ParameterDirection.Output ? " out" : "";
        if (stringBuilder == null)
        {                    
            stringBuilder = new StringBuilder(string.Format(CultureInfo.CurrentCulture
                    , "EXEC {0} {1}={{{2}}}{3}"
                    , StoredProcedureName
                    , param.ParameterName
                    , paramNum++
                    , direction));
        }
        else
        {
            stringBuilder.Append(string.Format(CultureInfo.CurrentCulture
                , ",{0}={{{1}}}{2}"
                , param.ParameterName
                , paramNum++
                , direction));
        }
    }

    try
    {
        _context.Database.ExecuteSqlRaw(
            stringBuilder.ToString()
            ,parameters.ToArray());
    }
    catch (Exception ex)
    {
         throw;
    }
}

The method take a List of SqlParameters in any order. Note that the list is converted to a SqlParamater array before being passed to ExecuteSqlRaw. It is not technically a "param array" but I have yet to see it fail. You may want to pass your DBContext or DatabaseFacade to the method as well.

I typically construct the list of parameters in the following fashion. The ParameterName property must match the name of the stored procedure parameter that it will feed. I've excluded any logic for inclusion/exclusion of parameters.

List<SqlParameter> parameters = new List<SqlParameter>();

parameters.Add(new SqlParameter()
    {
        ParameterName = "@MyOutputParam",
        DbType = System.Data.DbType.Int64,
        Direction = System.Data.ParameterDirection.Output
    });

parameters.Add(new SqlParameter()
    {
        ParameterName = "@RequiredButNullableUnicodeString",
        DbType = System.Data.DbType.String,
        Size = 50,
        Value = DBNull.Value
    });

parameters.Add(new SqlParameter()
   {
        ParameterName = "@SomeVarchar",
        DbType = System.Data.DbType.AnsiString,
        Size = 100,
        Value = "some value"
    });