简体   繁体   中英

double addition result being rounded

I have some code that is behaving strangely and seems to be rounding the result of the addition of two double values. This is causing issues with my code.

Unfortunately I cannot fix this issue because my unit testing environment is working OK (not rounding) and my application is not OK (rounding).

In my test environment:

a = -3.7468700408935547
b = 525218.0
c = b + a
c = 525214.25312995911

Inside my application:

a = -3.7468700408935547
b = 525218.0
c = b + a
c = 525214.25

What can be causing this? Project config? (I'm using visual studio, btw)

Edit (from comments)
I'm stepping through the same code using the Visual Studio debugger, so it's the exact same piece of code.

I have more code but I narrowed the problem down to that particular sum.

The binary representations of each value are:

test environment:

System.BitConverter.ToString(System.BitConverter.GetBytes(a))   "00-00-00-00-97-F9-0D-C0"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(b))   "00-00-00-00-44-07-20-41"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(c))   "00-40-9A-81-3C-07-20-41"   string

inside application:

System.BitConverter.ToString(System.BitConverter.GetBytes(a))   "00-00-00-00-97-F9-0D-C0"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(b))   "00-00-00-00-44-07-20-41"   string
System.BitConverter.ToString(System.BitConverter.GetBytes(c))   "00-00-00-80-3C-07-20-41"   string

Edit 2:

As Alexei Levenkov points out, this issue is caused by a library that changes the FPU config.

For anyone who is curious what this meant for me: I was able to mitigate this issue for my particular piece of code, by making some assumptions about my input values and doing some preemptive values rounding which in turn made my calculations consistent.

Use decimal if one wants preciscion such as needed in financial calculations.

Edit See:

Your application may be doing something strange with configuration of FPU. Ie using some random library for math which reconfigures precision...

Direct3d is possible suspect, see for example Pow implementation for double .

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM