简体   繁体   中英

Arithmetic operation with numeric datatype in SQL server yields different results

I get different results when using real and numeric data type. When I use real as datatype I get finalValue as -139.2466, when I use numeric datatype I get finalVaue as --139.246409. Which value is correct? When I plug these numbers in Excel, it matches to value -139.2466. For .eg

create table #resr ( a1 real, a2 real, a3 real)
insert #resr select  0.471163361822717, 0.0096160000 , 0.001669000000000
select a1*a2*-51.295/a3 finalValue from #resr

create table #resn ( a1 numeric(30,15), a2 numeric(30,15), a3 numeric(30,15))
insert #resn select  0.471163361822717, 0.0096160000 , 0.001669000000000
select a1*a2*-51.295/a3 finalValue from #resn

Floating point data types (of which REAL is a member) are approximate values, and can use any of a number of algorithms to encode the sequence of number, causing minute differences in how they're interpreted in SQL. This is the reason you can have a single float(10) value of 1234567890 and .1234567890

select cast(1234567890 as float(10))
select cast(.1234567890 as float(10))

Exact values (such as Decimal and Numeric) define exactly how many decimal places are allowed, and fills in zeroes for any out to as many as have been defined.

Floats give you the ability to model a wider range of numbers since you can allow extremely large numbers and extremely small numbers by allowing the decimal point to "float" rather than be a fixed point in memory. They're also fine in most cases as usually the decimal precision you lose isn't a big deal. They also tend to be smaller than precise data types (not always). However, if you know the size of the values you're expecting ahead of time, it's usually best to use a decimal.

Which value is "correct"? The numeric value. If you're ever comparing a floating point representation of a number vs an exact representation, go with the exact representation.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM