使用多线程读取和一个线程写入内存数据库

Question

I currently am working on a spring application which uses an in-memory database. The application receives many orders via MQ and it checks if the orders exists in the db or not. If it doesnt exist then after processing the order (business logic) it saves it to the db. Currently I am using a single thread to to handle these orders (read operation to check if already exist in db + business logic). Only when I want to save the order to the db (write operation) I hand off the task to another thread. So while the db is being checked to see if Order n + 10 exists already, another thread might be saving Order n to the db. When I increase the incoming rate for the orders I notice that certain read operations take a very long amount of time. I am wondering what could be the reason for this? Right now 2 threads are interacting with the db

1. thread 1 (read operation + business logic)
2. thread 2 (write operation)

I have already created indexes. Do I need to make sure both threads create a new session instead of using the current session? Can write and read happen concurrently? I am planning to increase the number of reader threads so i can process the orders faster. Its alright to insert the orders into db with some delay. Also Order is a hibernate entity.

Below is the code for reading from db

    @Transactional(propagation = Propagation.REQUIRED, readOnly = true)
    public Order findByOrderId(String Id, boolean isDeleted) {
        Session session = Objects.requireNonNull(getSessionFactory()).openSession();
        final List<Order> resultList = session
            .createQuery("from Order o where o.Id = :Id and isDeleted = :isDeleted", Order.class)
            .setParameter("Id", Id)
            .setParameter("isDeleted", isDeleted)
            .list();
        session.close();

        if (resultList.isEmpty()) {
            return null;
        }
        return (resultList.get(0));
    }

What changes can I make to my hibernate/JDBC properties for better performance?

Answer 1

There is no one single answer to improve the performance as it depends on a lot of factors. Having insights into the following factors will help us to understand why there is a performance bottleneck.

1. Checking the thread pool settings for the spring boot application
2. Checking the amount of CPU/Threads that are available on the server on which the spring boot app is running.

As the read operations are taking a lot of time, there are certain parameters that we need to look into to answer this

1. For hibernate, the number of round trips that we are making to fetch the data from the database.

2. The amount of data being fetched in each call from the database and amount of time it takes in transforming the data.

3. The Hibernate connection pool settings as we need to check the total number of open connections to the database.

4. If the data is in a sharded database, then we need to check the amount of time it takes in fetching the data from multiple shards.

As mentioned above, we need to look at multiple areas to check where we have the performance bottleneck.

Please also note that spawning a lot of threads will not help always as a lot of time is spent by the executor service in context switching and many threads are simply waiting for their turn to get executed.

I would recommend you to go through the article by Vlad Mihalcea https://vladmihalcea.com/hibernate-performance-tuning-tips/ where he explains all these things in detail.