SQL Server stored procedure - invalid column name

Question

I create a staging table in C#, fill it from a file, and then (in C# using SqlCommand ) run MyStoredProcedure which cleans and validates the data before adding it to the database tables.

The staging table that gets created is slightly different each time, because I only declare columns that exist in the file, so each time it has different fields depending on input. The stored procedure deals with it by running each piece of validation in a statement like this:

IF EXISTS(SELECT 1 FROM sys.columns WHERE Object_ID = Object_ID(N'stagingTable') AND Name = N'FirstName')
BEGIN
    UPDATE stagingTable 
    SET errors = ISNULL(errors, '') + 'First name column is blank.' 
    WHERE NULLIF(FirstName, '') IS NULL

    UPDATE stagingTable 
    SET FirstName = UPPER(FirstName)
END 

In order to compile and run the stored procedure the first time (ie when I'm working on it), I drop stagingTable and create it with ALL the possible columns, then run MyStoredProcedure once. After that, even when I drop stagingTable and recreate it with some columns left out, MyStoredProcedure still runs successfully - assuming this is because it compiled once and the IF statements are doing their job.

However, if I try doing this from my C# application, it throws an exception

Invalid Column Name

on the columns that don't exist in the current version of stagingTable . This seems strange - it overlooks the missing column names when it runs in SQL Server Management Studio, so long as it successfully compiled the first time after it was altered.

But from C#, it seems to be running a fresh version and checking each column name, therefore giving the "Invalid column name" error.

Why would it run just fine in SSMS, ignoring the invalid columns due to the if, and call this error when run from C#? Something fundamental about the compilation that I'm not understanding?

Answer 1

It is odd that the behaviour is different between C# and SSMS, as you state. I would actually expect it to always fail if the column doesn't exist at the time the server is parsing the query.

I assume that if you drop and recreate the stored procedure on a "clean" version of the staging table, it fails when run in SSMS?

One way around the compiler is to run the "dynamic" bit of the SQL as dynamic sql.

EXEC sp_executesql N'update stagingTable set FirstName = upper(FirstName)'

This is only a last resort though, and doesn't explain why you would be seeing the difference between the 2 environments.

Answer 2

Yes, this is a known behavior. When a row source (table or view) is referenced but missing as a whole, the parser will tolerate it; when a table or view does exist but is missing a column referenced anywhere in the statement, it will abort with an error. There is no clean way to turn deferred name resolution on for just a part of a conditional statement. Some tricks do exist; it is a matter of opinion if its worth going that extra mile, or just use dynamic (literal) commands instead:

• extract conditional part to a stored procedure and crank it around until it falls into submission: https://stackoverflow.com/a/4315884/1132334

• a summary of known approaches in connection with this problem: https://dba.stackexchange.com/a/24842/114522

The safe way home:

Use the EXEC('command-as-a-string') syntax as a workaround, for the part inside the EXISTS clause, doubling any single quotes inside:

IF EXISTS(SELECT 1 from sys.columns where Object_ID = Object_ID(N'stagingTable') AND [Name] = N'FirstName')
BEGIN
    EXEC('
        UPDATE stagingTable SET errors = ISNULL(errors, '''') + ''First name column is blank.'' WHERE NULLIF(FirstName, '''') IS NULL
        UPDATE stagingTable SET FirstName = UPPER(FirstName)
    ')
END