SQL Server 2008 R2 data type conversion error

Question

I have a column of data type varchar , some of which contain numerical (decimal) values. I need to find all rows with numerical values > 100. I'm using ISNUMERIC to filter rows that contain numbers, after which I'm trying to use CAST or CONVERT to change the numbers to a decimal value. I read another suggestion that said using a sub-query would fix the problem but I've tried that as well with no luck

This is the error message I'm receiving:

Arithmetic overflow error converting varchar to data type numeric.

Code:

select field_name
from table_name
where ISNUMERIC(field_name) = 1  and
      field_name in (select cast(field_name as decimal)
                     from table_name
                     where ISNUMERIC(field_name) = 1
                       and field_name not like '%,%'
                       and field_name not like '%-%'
                       and field_name != '.'
                       and field_name > 100.00)

Answer 1

This is your query:

select field_name
from table_name
where ISNUMERIC(field_name) = 1  and
      field_name in (select cast(field_name as decimal)
                     from table_name
                     where ISNUMERIC(field_name) = 1 and
                          field_name not like '%,%' and
                          field_name not like '%-%' and
                          field_name <> '.' and
                          field_name > 100
                    )

You are making the assumption that the where clause is filtered before the select . This is not true. In fact, SQL Server pushes the cast() to the operation that reads the data. It then applies the filter afterwards.

In fact, subqueries and common table expressions also do not guarantee the order of evaluation. What does is case (at least in this case that involves no aggregation). So, you can write your query as:

select field_name
from table_name
where (case when ISNUMERIC(field_name) = 1 and
                 field_name not like '%,%' and
                 field_name not like '%-%' and
                 field_name <> '.'
            then cast(field_name as decimal)
       end) > 100;

I see no reason for a subquery.

Answer 2

There are a few problems with the query, but the main problem, the one that is causing the error, is fairly simple: your field_name > 100.00 condition is not converting the VARCHAR field to a numeric type prior to doing the comparison. Changing it to CONVERT(DECIMAL(38, 18), [field_name]) would fix the error that you are seeing.

Run the following once to see that it does work, then uncomment the AND [field_name] > 100.00 line and run again and you will get the error you have been seeing about "Arithmetic overflow error converting varchar to data type numeric":

DECLARE @table_name TABLE (field_name VARCHAR(50) NOT NULL);
INSERT INTO @table_name (field_name) VALUES ('1234.5678');
INSERT INTO @table_name (field_name) VALUES ('12.78');
INSERT INTO @table_name (field_name) VALUES ('s');
INSERT INTO @table_name (field_name) VALUES ('2015-01-30');
INSERT INTO @table_name (field_name) VALUES ('51234.5678');
INSERT INTO @table_name (field_name) VALUES ('987651234.56789124');

SELECT field_name
FROM   @table_name
WHERE  ISNUMERIC([field_name]) = 1
AND    [field_name] IN (
                      SELECT CAST([field_name] AS DECIMAL)
                      FROM   @table_name
                      WHERE  ISNUMERIC([field_name]) = 1
                      AND    [field_name] NOT LIKE '%,%'
                      AND    [field_name] NOT LIKE '%-%'
                      AND    [field_name] != '.'
--                      AND    [field_name] > 100.00
                      AND    CONVERT(DECIMAL(38, 18), [field_name]) > 100.00
                     );

BUT , there are still some issues:

1. The subquery, as shown in the question, makes no sense to return a list of converted values. It is possible that the query has been altered for the purpose of posting here and so makes more sense in its true form, but the subquery isn't needed and isn't helping anyway.
2. The CAST to DECIMAL is not specifying the Precision and Scale. The default values are 18 and 0, respectively. Meaning, you are casting the strings into a DECIMAL(18, 0) . That does not appear to have any impact here, but it is best to specify the values. Running SELECT CAST('123.456' AS DECIMAL) will return just 123 .
3. You don't really need to filter out values with commas as those can be removed, leaving you with a value that can be converted. Unless, of course, you have some that are codes like 12,34,56,78 . But if they are valid numbers, such as 1,234.56 , then you can use REPLACE([field_name], ',', '')

4. Adding the following two lines to the INSERTs at the top of the example above will error:

    INSERT INTO @table_name (field_name) VALUES ('1,234.5678'); INSERT INTO @table_name (field_name) VALUES ('.'); 

All of this can be handled by doing the following (replace the query above with the one below, but keep the DECLARE and INSERT s):

SELECT field_name, CAST([field_name] AS DECIMAL(38, 18)) AS [DecimalValue]
FROM   @table_name
WHERE  ISNUMERIC(field_name) = 1
AND    field_name NOT LIKE '%,%'
AND    field_name NOT LIKE '%-%'
AND    field_name <> '.'
AND    CONVERT(DECIMAL(38, 18), REPLACE([field_name], ',', '')) > 100.00;