普及型VAccess控件v双重数据类型

Question

Hope someone can help! We're encountering an issue with the Pervasive VAccess control whereby any time we save an item of type 'double' into a Pervasive database, the value saved is different to the one we want...

As an example we try to save 1.44 and it actually saves 1.44004035454

A slight difference but a difference nonetheless!

FYI the field defined in the DDF has decimal set to 0, i'm wondering if one course of action is to set this to eg 4? But thought i'd see if anyone can shed any light on it before we head down that path...

Answer 1

The underlying effect is nothing to do with pervasive, it's a simple floating point issue . You'll find the same in any system that uses single- or double-precision floating point, though some systems do automatic rounding to hide this from you.

See http://en.wikipedia.org/wiki/Floating_point

In the case of PostgreSQL and its derivatives, you can set extra_float_digits to control this rounding.

regress=> SET extra_float_digits = 3;
SET
regress=> SELECT FLOAT8 '1.44';
       float8        
---------------------
 1.43999999999999995
(1 row)

regress=> SET extra_float_digits = 0;
SET
regress=> SELECT FLOAT8 '1.44';
 float8 
--------
   1.44
(1 row)

It defaults to 0, but your client driver might be changing it. If you're using JDBC (which I'm guessing you are) then don't mess with this setting, the JDBC driver expects it to remain how the driver sets it and will get upset at you if you change it.

In general, if you want a human-readable formatted number you should be doing the rounding with round or to_char , or doing it client-side, instead. Note that there's no round(double precision, integer) function for reasons explained in answers to this question . So you'll probably want to_char , eg.

regress=> SELECT to_char(FLOAT8 '1.44', 'MI999999999D99');
    to_char    
---------------
          1.44
(1 row)

(I wish PostgreSQL exposed a version of the cast from float8 to text that let you specify extra_float_digits on a per-call basis. That's often closer to what people really want. Guess I should add that if I get the time...)