Rounding issue when dividing a COUNT value by another value using jooq

Question

When using SQL, I can run a simple query such as the query below with no issue, it returns an answer to 4 decimal places:

SELECT(COUNT(ID)/7) FROM myTable;

If the count above returns a value of 12, the returned select value is 1.7143 in workbench.

My issue occurs when I use jooq to do this calculation:

dsl.select(count(MYTABLE.ID).divide(7).from(MYTABLE).fetch();

The above code returns me a value of 1, whereas I want a value of 1.7143.

I have similar lines of jooq code which use SUM as opposed to COUNT and they return a value to 4 decimal places but I cannot find a way to get the above code to return the value to 4 decimal places.

I have tried using .round but had no success.

Has anyone else had a similar problem and knows of a solution?

Answer 1

There are two issues here, depending on what RDBMS you're using:

1. The type of the whole projected expression

The type of the whole division expression depends on the type of the left hand side (dividend), which is SQLDataType.INTEGER . So, irrespective of whether your RDBMS returns a decimal or floating point number, jOOQ will use JDBC's ResultSet.getInt() method to fetch the value, where you will be losing precision. So, the first step is to make sure jOOQ will fetch the desired data type. There are several ways to do this:

1. Use a cast on the COUNT(*) expression: count(MYTABLE.ID).cast(SQLDataType.DOUBLE).divide(7)
2. Use a cast on the entire expression: count(MYTABLE.ID).divide(7).cast(SQLDataType.DOUBLE)
3. Use data type coercion on either expression: expr.coerce(SQLDataType.DOUBLE)

Casts have an effect on the generated SQL. Data type coercions do not.

2. How your RDBMS handles data types

In most RDBMS, count(*) expressions produce an integer type, and your division's right hand side (divisor) is also an integer, so the best resulting data type is, in fact, an integer type. I suspect you should pass a double or BigDecimal type as your divisor instead.

Solution

The ideal solution would then be to combine the above two:

dsl.select(count(MYTABLE.ID).cast(SQLDataType.DOUBLE).divide(7.0))
   .from(MYTABLE)
   .fetch();