big.Float SetPrec怪异的行为

Question

After doing some calculations using big.Float in golang, I am setting the precision to 2.

And even thou the number is just a simple 10, after setting the precision it is 8.

package main

import (
    "fmt"
    "math/big"
)

func main() {
    cost := big.NewFloat(10)
    fmt.Println("COST NOW", cost)

    perKWh := big.NewFloat(0)
    cost.Add(cost, perKWh)
    fmt.Println("COST ", cost.String())

    perMinute := big.NewFloat(0)
    cost.Add(cost, perMinute)
    fmt.Println("COST ", cost.String())

    discountAmount := big.NewFloat(0)
    cost.Sub(cost, discountAmount)
    floatCos, _ := cost.Float64()
    fmt.Println(fmt.Sprintf("COST FLOAT %v", floatCos))
    cost.SetPrec(2)

    fmt.Println("COST ", cost.String())
}

Check playground example here: https://play.golang.org/p/JmCRXkD5u49

Would like to understand why

Answer 1

From the fine manual :

type Float
[...]
Each Float value also has a precision, rounding mode, and accuracy. The precision is the maximum number of mantissa bits available to represent the value. The rounding mode specifies how a result should be rounded to fit into the mantissa bits, and accuracy describes the rounding error with respect to the exact result.

And big.Float is represented internally as:

sign × mantissa × 2**exponent

When you call SetPrec you're setting the number of bits available for the mantissa, not the number of digits of precision in the decimal representation of the number.

You can't represent decimal 10 (1010 binary) in two bits of mantissa so it rounds to decimal 8 (1000 binary) which can fit into 2 bits. You need at least three bits to store the 101 part of decimal 10. 8 can fit into a single bit of mantissa so you'll see the same 8 if you say cost.SetPrec(1) .

You need to be thinking in terms of binary when using the big package.

Answer 2

First, discard all the irrelevant code. Next, print useful diagnostic information.

package main

import (
    "fmt"
    "math/big"
)

func main() {
    cost := big.NewFloat(10)
    fmt.Println("Cost ", cost.String())
    fmt.Println("Prec", cost.Prec())
    fmt.Println("MinPrec", cost.MinPrec())
    fmt.Println("Mode", cost.Mode())
    cost.SetPrec(2)
    fmt.Println("Prec", cost.Prec())
    fmt.Println("Accuracy", cost.Acc())
    fmt.Println("Cost ", cost.String())
}

Output:

Cost  10
Prec 53
MinPrec 3
Mode ToNearestEven
Prec 2
Accuracy Below
Cost  8

Round 10 to the nearest even number that can be represented in an sign, exponent, and a 2-bit mantissa and you get 8.

Rounding ToNearestEven is IEE754 rounding. Round to nearest, ties to even – rounds to the nearest value; if the number falls midway it is rounded to the nearest value with an even (zero) least significant bit.