query with negation of AND and NOT EXIST for SSIS Lookup

Question

I've been piecing together SQL queries to check a table if a record exists using 2 comparisons and a "NOT EXIST". The SQL is in the 'Custom query' of the Lookup Transform (under Advanced tab). Then the "Lookup Match Output" goes to a DB and the "Lookup No Match Output" goes to a flat file.

SELECT *
FROM [dbo].[MyTable] 
 WHERE NOT EXISTS (SELECT 1
    FROM [dbo].[MyTable] 
      WHERE [dbo].[MyTable].[DateStamp] = ?
      AND [dbo].[MyTable].[SomeID] = ?
)

However the "NOT" doesn't change the outcome. All records go to the "Lookup No Match Output". I had expected to see the output change to the other branch when removing the NOT. What's wrong with this query?

It's probably simple but driving me nuts. Please help.

Additional info: The table is initially empty. The purpose of the query is to populate the table. And this query's checks is to prevent duplicate entries.The values compared against ("?") are always non null.

Answer 1

The behavior you describe would happen if the comparisons included NULL values. One possibility is that ? represents a NULL value. Then the conditions would always return NULL . And, NOT NULL is just as not-true as NULL .

Another possibility is that there are no matching rows for legitimate inputs for ? . And, one or both columns are sometimes NULL . This would cause the same issue.

You could fix this second problem by adding where DateStamp is not null and SomeId is not null to the subquery.

Answer 2

The answer was provided by Srinivasa Rao Gude in another forum. He stated:

Lookup Transformation will always do a equi- join between the input and a look up reference data set. As we don't have a rows initially in [My Table] inner join is failing hence its redirecting to Lookup No Match Output direction. ... update your query to EXISTS instead of NOT EXISTS ... update the lookup no match output clause to DB and matched clause to Flat file.

This worked. Thanks Srinivasa!