Error: Arithmetic overflow error converting varchar to data type numeric

Question

select distinct f1.[record] from [field] f1 where ((ABS((Cast((Select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay1_read1')) as Decimal(5,3))) - (Cast((select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay1_read2')) as Decimal(5,3)))) > 1.0) 
or (ABS((Cast((Select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay2_read1')) as Decimal(5,3))) - (Cast((select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay2_read2')) as Decimal(5,3)))) > 1.0) 
or (ABS((Cast((Select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay3_read1')) as Decimal(5,3))) - (Cast((select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay3_read2')) as Decimal(5,3)))) > 1.0) 
or (ABS((Cast((Select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay4_read1')) as Decimal(5,3))) - (Cast((select [value] from dbo.getFieldTopTable(f1.[record], 'ar_essay4_read2')) as Decimal(5,3)))) > 1.0))

The data stored in these fields is setup at real and ranges in value from .001 to 4.00 (should it be int?)

We're just trying to find records where the difference between 2 scores is greater than 1

We've also just tried the cast _ to decimal with no precision and it leaves out records that should be in the query...

Is there something we're not thinking about?

Answer 1

You shouldn't need to cast these values at all, the problem is most likely the datatype of the value returned by your dbo.getFieldTopTable() function. Ensure that it is returning a value with decimals.

Answer 2

Digits to the right of the decimal place should not cause a truncation error. Try DECIMAL(9,3). That consumes the same memory as DECIMAL(5,3).