Discrepancy between floats in C and Python

Question

I am making an implementation of a MobileNetV2 in C and comparing it, layer by layer, to a Keras model of the same network to make sure I'm doing things right. I managed to get to a very close approximation of the inference result of the network, but there is an error in the 5th decimal place or so. Looking for the reason for the imprecision I came across something strange.

I am working exclusively with float objects in C, and all of the arrays in Python, including all of the weight arrays and other parameters, are float32 .

When exporting my processed image from Python to a .csv file, I put seven decimal points to the export function: np.savetxt(outfile, twoD_data_slice, fmt='%-1.7e') which would still result in a float but with certain limitations. Namely, that last decimal place does not have full precision. However, one of the numbers I got was "0.98431373". When trying to convert this to C it instead gave me "0.98431377".

I asked a question here about this result and I was told of my mistake to use seven decimal places, but this still doesn't explain why Python can handle a number like "0.98431373" as a float32 when in C that gets changed to "0.98431377".

My guess is that Python is using a different 32-bit float than the one I'm using in C, as evidenced by how their float32 can handle a number like "0.98431373" and the float in C cannot. And I think this is what is causing the imprecision of my implementation when compared to the final result in Python. Because if Python can handle numbers like these, then the precision it has while doing calculations for the neural network is higher than in C, or at least different, so the answer should be different as well.

Is the floating point standard different in Python compared to C? And if so, is there a way I can tell Python to use the same format as the one in C?

Answer 1

A 32-bit floating point number can encode about 2 ³² different values.
0.98431373 is not one of them.
Finite floating point values are of the form: some_integer * power-of-two .

The closest choice to 0.98431373 is 0.98431372_64251708984375 which is 16514044 * 2 ^-24 .

Printing 0.98431372_64251708984375 to 8 fractional decimal places is 0.98431373 . That may appear to be the 32-bit float value, but its exact value differs a small amount.

---

in C that gets changed to "0.98431377"

0.98431377 is not an expected output of a 32-bit float as the next larger float is 0.98431378_6029815673828125. Certainly OP's conversion code to C results in a 64-bit double with some unposted TBD conversion artifacts.

"the way I import the data to C is I take the mantissa, convert it to float, then take then exponent, convert it to long, and then multiply the mantissa by 10^exponent" is too vague. Best to post code than only a description of code.

---

Is the floating point standard different in Python compared to C?

They could differ, but likely are the same.

And if so, is there a way I can tell Python to use the same format as the one in C?

Not really. More likely the other way around. I am certain C allows more variations on FP than python.