将Nvarchar转换为Int失败的SQL Server 2008

Question

I have some rows I'm trying to cast to integer to get the last number in sequence.

This is my original query.

SELECT
  MAX(CAST(REPLACE(ItemName, 'CA', '') AS INT)) + 1
FROM InventoryItem ii
JOIN InventoryItemDepartment iid
  ON ii.ItemCode = iid.ItemCode
WHERE iid.DepartmentCode = 'Filters'
AND ItemName LIKE 'CA4%____'
AND CAST(REPLACE(ItemName, 'CA', '') AS INT) < 41000

However I get an error message: Error (1,1): Conversion failed when converting the nvarchar value '41020-S' to data type int.

Obviously, I'm aware what this message means. However I'm confused why its throwing the error, as the WHERE clauses I've specified are for the sole purpose of excluding records which might fail the cast.

If I modify the query to just select the original values, without any replacing or casting...

SELECT
  ItemName
FROM InventoryItem ii
JOIN InventoryItemDepartment iid
  ON ii.ItemCode = iid.ItemCode
WHERE iid.DepartmentCode = 'Filters'
AND ItemName LIKE 'CA4%____'

This returns some data like so:

CA40000
CA40001
CA40002
CA40003
CA40004
CA40005
.... etc

The troublesome value '41020-S' (Originally 'CA41020-S' after the Replace) isn't included in the second debug result set, as I'd expected.

Can anyone help me explain this weird behaviour, and how I might possibly overcome it?

Answer 1

There's no guarantee on the evaluation order of individual predicates in the WHERE clause ^* . (Nor does SQL Server guarantee not to evaluate expressions in the SELECT clause for values that should be filtered by the WHERE clause).

The strongest way to guarantee that your filters take effect is, unfortunately, to split your query into two separate queries - your first query performing the required filtering and putting its results into a temp table/table variable and your second query building from that and performing data conversions. ¹

The slightly weaker way that almost always works, except sometimes with aggregates it can be a bit funny ² is to use CASE expressions instead:

SELECT
  MAX(CASE WHEN iid.DepartmentCode = 'Filters'
AND ItemName LIKE 'CA4%____'
THEN CAST(REPLACE(ItemName, 'CA', '') AS INT)
ELSE 60000 END) + 1
FROM InventoryItem ii
JOIN InventoryItemDepartment iid
  ON ii.ItemCode = iid.ItemCode
WHERE CASE WHEN iid.DepartmentCode = 'Filters'
AND ItemName LIKE 'CA4%____'
THEN CAST(REPLACE(ItemName, 'CA', '') AS INT)
ELSE 60000 END < 41000

---

¹ This goes against the usual advice to build a single large query and let the optimizer find the best way of evaluating your query. Unfortunately, the optimizer often gets this one wrong and there's no indication that Microsoft plan to fix this since it's been a known issue for more than a decade ⁺ .

Note that any answer purporting to fix this just by rearranging the query (such as placing parts into subqueries) or adding additional guard clauses may superficially appear to fix the problem by accidentally forcing the optimizer to choose a different plan. But you've got no guarantee over if or when the optimizer will return to using a plan that does generate the error message again.

² CASE : "In some situations, an expression is evaluated before a CASE statement receives the results of the expression as its input. Errors in evaluating these expressions are possible. Aggregate expressions that appear in WHEN arguments to a CASE statement are evaluated first, then provided to the CASE statement."

^* Unlike some other programming languages, SQL offers no guarantee such as left-to-right evaluation, nor any way to influence whether or not it exhibits any short-circuiting behaviour.

⁺ This issue was originally reported on User Voice. Unfortunately, in it's migration to the Azure feedback forum a lot of detail has been compressed into the single Microsoft "response" which makes it difficult to read, and "oh dear" also lost the substantial votes it had previously received on User Voise.

Answer 2

Try this query, change the condition in where clause to check only four characters after CA4

SELECT
  MAX(CAST(REPLACE(ItemName, 'CA', '') AS INT)) + 1
FROM InventoryItem ii
JOIN InventoryItemDepartment iid
  ON ii.ItemCode = iid.ItemCode
WHERE iid.DepartmentCode = 'Filters'
AND ItemName LIKE 'CA4____'
AND CAST(REPLACE(ItemName, 'CA', '') AS INT) < 41000

Still above query may fail if execution plan chose to evaluate the condition CAST(REPLACE(ItemName, 'CA', '') AS INT) < 41000 first. To be fail safe you can use below query.

SELECT
  MAX(CAST(REPLACE(ItemName, 'CA', '') AS INT)) + 1
FROM 
(   SELECT ItemName
    FROM InventoryItem ii
    JOIN InventoryItemDepartment iid
      ON ii.ItemCode = iid.ItemCode
    WHERE iid.DepartmentCode = 'Filters'
    AND ItemName LIKE 'CA4____'
) AS SubQ
WHERE CAST(REPLACE(ItemName, 'CA', '') AS INT) < 41000

Answer 3

There are two learning points here:

1. Avoid using CAST and CONVERT in WHERE expressions wherever possible - they fragilize the query and reduce performance due to performing conversions on rows which will be excluded.

2. MAX() also works with string values.

Assuming the values from which you want to find the maximum are in the range CA40000 to CA40999 and the data are well formed, apart from the occasional suffix like the 41020-S which broke your query, you could use :

SELECT CAST(MAX(SUBSTRING(ItemName, 3, 5)) AS INT) + 1 FROM InventoryItem ii INNER JOIN InventoryItemDepartment iid ON ii.ItemCode = iid.ItemCode WHERE iid.DepartmentCode = 'Filters' AND ItemName LIKE 'CA40___%'

In a very large table, MAX(LEFT(ItemName, 7)) might be faster, because it can directly use an index on ItemName, but this makes the query a little more complicated.

If the data could have non-numeric values after CA40, one can use range matching with LIKE to avoid errors : LIKE 'CA40[0-9][0-9][0-9]%'

Answer 4

Use this

MAX(CAST(REPLACE(REPLACE(name, 'CA', ''),'-S','') AS INT)) + 1

In place of

MAX(CAST(REPLACE(ItemName, 'CA', '') AS INT)) + 1

And for

CAST(REPLACE(REPLACE(name, 'CA', ''),'-S','') AS INT)

in place

CAST(REPLACE(ItemName, 'CA', '') AS INT)

Answer 5

You can use ISNUMERIC if you want to remove any rows with unexpected values, such as having an unknown character in it.

SELECT
  MAX(CAST(REPLACE(ItemName, 'CA', '') AS INT)) + 1
FROM InventoryItem ii
JOIN InventoryItemDepartment iid
  ON ii.ItemCode = iid.ItemCode
WHERE iid.DepartmentCode = 'Filters'
AND ItemName LIKE 'CA4%____'
AND ISNUMERIC(CAST(REPLACE(ItemName, 'CA', '')) = 1 AND CAST(REPLACE(ItemName, 'CA', '') AS INT) < 41000.

Note: ISNUMERIC is not perfect. It will consider some characters as numeric too. You can read it here .