I am facing with 2 problems: N + 1 query and Out Of Memory (OOM).
I solved OOM by paging and lazy loading:
@OneToMany(fetch = FetchType.LAZY)
@JoinColumn(name = "department_id")
private Set<Employee> employees;
But when I use lazy loading, N + 1 query happened. So I try to use EntityGraph
as https://www.baeldung.com/spring-data-jpa-named-entity-graphs . But as my researches and local test, EntityGraph
always do eager loading for NamedAttributeNode
field - association field, which I want to be lazy loading - do not load all data at first:
@Entity
@Table(name = "department")
@NamedEntityGraph(name = "Department",
attributeNodes = {
@NamedAttributeNode("employees")
}
)
public class Department implements Serializable {
@OneToMany(fetch = FetchType.LAZY)
@JoinColumn(name = "department_id")
private Set<Employee> employees;
}
So are there any way to get them both? Use EntityGraph
to avoid N + 1 and lazy loading to avoid OOM?
UPDATE: Can EntityGraph
works fine with Pageable effectively? I mean do not load all data in JOIN query.
Using EntityGraph
all your NamedAttributeNode
associations will be loaded in 1 query with Join
clause. Enable sql log to see how many queries hibernate does for loading entities in different scenarios
logging.level.org.hibernate.SQL=DEBUG
You will see that using @OneToMany(fetch = FetchType.EAGER)
without EntityGraph
it loads employees in separate select
queries (N + 1), but using EntityGraph
it performs only 1 select ... join
Also don't forget to specify entity graph name in repository like:
@EntityGraph(value = "Department")
List<Department> findAll();
UPDATE: Spring DATA Pagination doesn't work on database side. It will fetch all data and then filtered in memory. That's how it works.. There are some workarounds, check this links:
How can I avoid the Warning "firstResult/maxResults specified with collection fetch; applying in memory!" when using Hibernate?
Avoiding "HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!" using Spring Data
VladMihalcea Blog The best way to fix the Hibernate HHH000104
As for me the solution could be creating custom repository and using EntityManager
to construct query manually.
A simple solution is:
@EntityGraph
.@EntityGraph
from previous returned ids.=> No N+1 complexity, only 2 SQL requests and no native queries, etc.
Example:
On your main repository, add @EntityGraph
annotation with associations you want to fetch:
public interface CarRepository extends JpaRepository<Car, String>, JpaSpecificationExecutor<Car>, CarCustomRepository {
@Override
@EntityGraph(attributePaths = { "foo", "bar", "bar.baz" })
List<Car> findAllById(Iterable<String> ids);
}
Create a custom repository with a findAllWithEntityGraph
method:
public interface CarCustomRepository {
Page<Car> findAllWithEntityGraph(Specification<Car> specification, Pageable pageable);
}
The implementation of the custom repository. It first fetch entities without entity graph, then it re-fetch them with entity graph in order to load associations. Don't forget to re-sort entities in to preserve order:
public class CarCustomRepositoryImpl implements CarCustomRepository {
@Autowired
@Lazy
private CarRepository carRepository;
@Override
public Page<Car> findAllWithEntityGraph(Specification<Car> specification, Pageable pageable) {
Page<Car> page = carRepository.findAll(specification, pageable);
List<String> ids = page.getContent().stream().map(Car::getId).collect(Collectors.toList());
List<Car> cars = carRepository.findAllById(ids).stream()
.sorted(Comparator.comparing(car -> ids.indexOf(car.getId())))
.collect(Collectors.toList());
return new PageImpl<>(cars, pageable, page.getTotalElements());
}
}
Then, you just have to invoke CarRepository#findAllWithEntityGraph
method in order to fetch records with pagination and no N+1 complexity.
The question is: why hibernate does not have this behavior by default?
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.