简体   繁体   中英

comparison float rounding fails System.Math.RoundTo C++ XE7

I've been trying to round a float value to 4 precision without success.

float fconv = 1.0f;
float fdata = 39.934543423412f;
float fres = RoundTo(fdata*fconv, -4);

if(fres <= 39.9345f){do something;} //<-- unwanted behavior

Wanted result is 39.934500000000
Actual result is 39.934543423412

I've tried many methods including Round a float to a given precision without success.

I'm working on an AMD FX83xx 64bit. Program is built in 32bit Debug using XE7

Thanks

Your desired precision of 6 decimal digits is very near the precision limits for a float data type. The epsilon, or delta between consecutive representable floating point values, for a number around 40f is about 7.63E-6, so there's only a couple of bits different between the 'best' value and what you're getting. This is possibly due to rounding that close to the limit, but I'm not sure.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM