Hibernate: projection JPQL query to DTO issue

Question

To start with, I'll list three models that I work with in a query

ProductEntity:

@Entity
@Table(name = "product")
public class ProductEntity extends BaseEntity {
  //some fields

   @ManyToOne(fetch = FetchType.LAZY)
   @JoinColumn(name = "owner_id")
   private PartnerEntity owner;

   @OneToMany(
            mappedBy = "product",
            fetch = FetchType.LAZY
    )
    private List<StockProductInfoEntity> stocks;
 }

PartnerEntity:

@Entity
@Table(name = "partner")
public class PartnerEntity extends AbstractDetails {
    //some fields

    @OneToMany(
            mappedBy = "owner",
            fetch = FetchType.LAZY
    )
    private List<ProductEntity> products;
 }

and StockProductInfoEntity:

@Entity
@Table(name = "stock_product")
public class StockProductInfoEntity extends BaseEntity {
    @ManyToOne(fetch = FetchType.LAZY)
    @JoinColumn(name = "product_id")
    private ProductEntity product;

    //other fields
    @Column(name = "rest")
    private int rest;
}

And i want to fetch from database product with partner + calculate count in all stocks. For convenience, I created a simple DTO:

@Getter
@AllArgsConstructor
public class ProductCountDTO {
    private ProductEntity productEntity;
    private int count;

    //hack for hibernate
    public ProductCountDTO(ProductEntity productEntity, long count) {
        this.productEntity = productEntity;
        this.count = (int) count;
    }
}

and write JPQL query in JPA repository:

@Query("select new ru.oral.market.persistence.entity.product.util.ProductCountDTO(p, sum(stocks.rest))"+ 
            " from ProductEntity p" +
            " join fetch p.owner owner" +
            " join p.stocks stocks" +
            " where p.id = :id" +
            " group by p, owner")
    Optional<ProductCountDTO> findProductWithCount(@Param("id") long id);

But my application did not even start because of a problem with the query validation. I get this message:

Caused by: org.hibernate.QueryException: query specified join fetching, but the owner of the fetched association was not present in the select list

Very strange, but I tried to replace join fetch -> join. And I understood why I got this error, hibernate made such a query to the database:

select
            productent0_.id as col_0_0_,
            sum(stocks2_.rest) as col_1_0_ 
        from
            product productent0_ 
        inner join
            partner partnerent1_ 
                on productent0_.owner_id=partnerent1_.user_id 
        inner join
            stock_product stocks2_ 
                on productent0_.id=stocks2_.product_id 
        where
            productent0_.id=? 
        group by
            productent0_.id ,
            partnerent1_.user_id

But why does he only take the product id and nothing else? This query with Tuple work and get all fields from product and partner

  @Query("select p, sum(stocks.rest) from ProductEntity p" +
            " join fetch p.owner owner" +
            " join p.stocks stocks" +
            " where p.id = :id" +
            " group by p, owner")
    Optional<Tuple> findProductWithCount(@Param("id") long id);

And this produced native query what i want:

select
            productent0_.id as col_0_0_,
            sum(stocks2_.rest) as col_1_0_,
            partnerent1_.user_id as user_id31_12_1_,
            productent0_.id as id1_14_0_,
            productent0_.brand_id as brand_i17_14_0_,
            productent0_.commission_volume as commissi2_14_0_,
            productent0_.created as created3_14_0_,
            productent0_.description as descript4_14_0_,
            productent0_.height as height5_14_0_,
            productent0_.length as length6_14_0_,
            productent0_.long_description as long_des7_14_0_,
            productent0_.name as name8_14_0_,
            productent0_.old_price as old_pric9_14_0_,
            productent0_.owner_id as owner_i18_14_0_,
            productent0_.pitctures as pitctur10_14_0_,
            productent0_.price as price11_14_0_,
            productent0_.status as status12_14_0_,
            productent0_.updated as updated13_14_0_,
            productent0_.vendor_code as vendor_14_14_0_,
            productent0_.weight as weight15_14_0_,
            productent0_.width as width16_14_0_,
            partnerent1_.about_company as about_co1_12_1_,
            partnerent1_.bik as bik2_12_1_,
            partnerent1_.bank_inn as bank_inn3_12_1_,
            partnerent1_.bank_kpp as bank_kpp4_12_1_,
            partnerent1_.bank as bank5_12_1_,
            partnerent1_.bank_address as bank_add6_12_1_,
            partnerent1_.checking_account as checking7_12_1_,
            partnerent1_.correspondent_account as correspo8_12_1_,
            partnerent1_.company_name as company_9_12_1_,
            partnerent1_.company_inn as company10_12_1_,
            partnerent1_.company_kpp as company11_12_1_,
            partnerent1_.ogrn as ogrn12_12_1_,
            partnerent1_.okato as okato13_12_1_,
            partnerent1_.actual_address as actual_14_12_1_,
            partnerent1_.director as directo15_12_1_,
            partnerent1_.full_name as full_na16_12_1_,
            partnerent1_.legal_address as legal_a17_12_1_,
            partnerent1_.short_name as short_n18_12_1_,
            partnerent1_.country as country19_12_1_,
            partnerent1_.discount_conditions as discoun20_12_1_,
            partnerent1_.discounts as discoun21_12_1_,
            partnerent1_.logo as logo22_12_1_,
            partnerent1_.min_amount_order as min_amo23_12_1_,
            partnerent1_.min_shipment as min_shi24_12_1_,
            partnerent1_.min_sum_order as min_sum25_12_1_,
            partnerent1_.own_delivery as own_del26_12_1_,
            partnerent1_.own_production as own_pro27_12_1_,
            partnerent1_.phones as phones28_12_1_,
            partnerent1_.return_information as return_29_12_1_,
            partnerent1_.site as site30_12_1_ 
        from
            product productent0_ 
        inner join
            partner partnerent1_ 
                on productent0_.owner_id=partnerent1_.user_id 
        inner join
            stock_product stocks2_ 
                on productent0_.id=stocks2_.product_id 
        where
            productent0_.id=? 
        group by
            productent0_.id ,
            partnerent1_.user_id

But it's not very convenient. Why DTO projection doesn't work correctrly, but tuple works fine?

Answer 1

Because that's how Hibernate is currently implemented.

Because you used an entity in the DTO Projection, which as the name implies, it should be used for DTOs, not entities, Hibernate is going to assume that you want to GROUP BY by the identifier because it should not GROUP BY all entity properties.

The Tuple is broken and it will only work in MySQL, but not in Oracle or PostgreSQL since your aggregate query selects columns that are not present in the GROUP BY clause.

However, this is not demanded to work according to the JPA specs. Nevertheless, you should still provide a replicating test case and open an issue so that the behavior is the same for both situations.

Anyway, once fixed, it will still GROUP BY identifier. If you want to select entities and group by as well, you will have to use a native SQL query along with the Hibernate ResultTransformer to transform the ResultSet into a graph of objects.

More, fetching entities and aggregations is a code smell. Most likely, you need a DTO projection or a read-only view.

Entities should only be fetched when you want to modify them. Otherwise, a DTO projection is more efficient and more straightforward as well.

Answer 2

Since Vlad already explained the why, I will focus on an alternative solution. Having to specify all attributes that you are really interested in in the SELECT clause and the GROUP BY clause is a lot of work. If you used Blaze-Persistence Entity Views on top of Hibernate, this could look like the following

@EntityView(ProductEntity.class)
public interface ProductCountDTO {
    // Or map the ProductEntity itself if you like..
    @Mapping("this")
    ProductView getProduct();
    @Mapping("sum(stocks.rest)")
    int getCount();
}

@EntityView(ProductEntity.class)
public interface ProductView {
    // Whatever mappings you like
}

With the Spring Data or DeltaSpike Data integration you can even use it like that

Optional<ProductCountDTO> findById(long id);

It will produce a JPQL query like the following

SELECT
  p /* All the attributes you map in ProductView  */,
  sum(stocks_1.rest)
FROM
  ProductEntity p
LEFT JOIN
  p.stocks stocks_1
GROUP BY
  p /* All the attributes you map in ProductView */

Maybe give it a shot? https://github.com/Blazebit/blaze-persistence#entity-view-usage

The magic is that Blaze-Persistence handles the GROUP BY automatically when encountering an aggregate function by putting every non-aggregate expression you use into the GROUP BY clause if there is at least one aggregate function used. When using Entity Views instead of entities directly, you won't be facing the join fetch problems as Entity Views will only put the fields you actually map into the resulting SELECT clause of the JPQL and SQL. Even if you used entities directly or via the ProductCountDTO, the query builder used behind the scenes handles selects of entity types in case of a group by gracefully, just as you'd expect it from Hibernate.