BeanPropertyRowMapper上的Java慢速SQL检索到ArrayList <accountQueryResult>

Question

we have a very slow Java API function in SpringBoot framework, and follows is a profiling output of it:

We believe this is coming from this line of code:

The query before that line, retrieves 41,573 rows (given specified accountId and one more parameter) - this is done in acceptable time of about 2 secs.

return (ArrayList) this.pbaJdbc.query(sql, new Object[] { accountId, taxRegId }, bpm);

We tried to set the FetchSize to 1000 (it was set to -1 by default which I was unable to determine the meaning of in documentation) - this change did not improve the performance of the operation noticeably.

One other suggestion was to switch to a HashMap structure instead of ArrayList, but I am not sure how to do that... tried to write a hash map, something like:

HashMap<Integer, accountQueryResult> accmap = new HashMap(Integer, accountQueryResult);

But this errors with Expression expected, I am not sure how to do it right.

This is the relevant piece of code for your review:

@Repository
public class UserRepository {
    @Autowired
    @Qualifier("pbaJdbc")
    private JdbcTemplate pbaJdbc;

    public ArrayList<accountQueryResult> getAccountsSubscriptionsResources(int accountId, String taxRegId) {

        BeanPropertyRowMapper bpm = new BeanPropertyRowMapper(accountQueryResult.class);
        bpm.setPrimitivesDefaultedForNullValue(true);
        String sql =

                "SELECT " +
                        "\"Account\".\"AccountID\",\n" +
                        "\"Account\".\"VendorAccountID\",\n" +
                        "\"Account\".\"AdminPhAreaCode\",\n" +
                        "\"Account\".\"AdminPhNumber\",\n" +
                        "\"Account\".\"AdminFaxAreaCode\",\n" +
                        "\"Account\".\"AdminFaxNumber\",\n" +
                        "\"Account\".\"AdminEmail\",\n" +
                        "\"Account\".\"PersPhAreaCode\",\n" +
                        "\"Account\".\"PersPhNumber\",\n" +
                        "\"Account\".\"PersFaxAreaCode\",\n" +
                        "\"Account\".\"PersFaxNumber\",\n" +
                        "\"Account\".\"PersEmail\",\n" +
                        "\"Account\".\"TaxStatus\",\n" +
                        "\"Account\".\"CompanyName\",\n" +
                        "\"Account\".\"Address1\",\n" +
                        "\"Account\".\"Address2\",\n" +
                        "\"Account\".\"City\",\n" +
                        "\"Account\".\"Zip\",\n" +
                        "\"ActivePaytool\".\"CutNumber\",\n" +
                        "\"ActivePaytool\".\"PaySystem\",\n" +
                        "\"Subscription\".\"subscriptionID\",\n" +
                        "\"Subscription\".\"SubscriptionName\",\n" +
                        "\"Subscription\".\"Status\",\n" +
                        "\"Subscription\".\"PlanID\",\n" +
                        "\"PlanPeriod\".\"Period\",\n" +
                        "\"PlanPeriod\".\"PlanPeriodID\",\n" +
                        "\"PlanPeriod\".\"PeriodType\",\n" +
                        "\"PlanPeriod\".\"RenewalFee\",\n" +
                        "\"PlanPeriod\".\"SetupFee\",\n" +
                        "\"SubscrParam\".\"resourceID\",\n" +
                        "\"BMResource\".\"name\" AS \"ResourceName\",\n" +
                        "\"SubscrParam\".\"IncludedValue\",\n" +
                        "\"SubscrParam\".\"Amount\",\n" +
                        "\"SubscrParamValue\".\"IdParameter\",\n" +
                        "\"SubscrParamValue\".\"Value\",\n" +
                        "\"IntUsers\".\"UsersID\",\n" +
                        "\"IntUsers\".\"Login\" AS \"LoginID\"\n" +
                        "FROM\n" +
                        "\"Account\"\n" +
                        "LEFT JOIN \"IntUsers\" ON \"IntUsers\".\"AccountID\" = \"Account\".\"AccountID\"\n" +
                        "LEFT JOIN \"Subscription\" ON \"Subscription\".\"AccountID\" = \"Account\".\"AccountID\"\n" +
                        "AND \"Subscription\".\"Status\" IN ('30', '40', '15')\n" +
                        "LEFT JOIN \"SubscrParam\" ON \"SubscrParam\".\"subscriptionID\" = \"Subscription\".\"subscriptionID\"\n" +
                        "LEFT JOIN \"BMResource\" ON \"BMResource\".\"resourceID\" = \"SubscrParam\".\"resourceID\"\n" +
                        "LEFT JOIN \"SubscrParamValue\" ON \"SubscrParamValue\".\"subscriptionID\" = \"Subscription\".\"subscriptionID\"\n" +
                        "LEFT JOIN \"PlanPeriod\" ON \"PlanPeriod\".\"PlanID\" = \"Subscription\".\"PlanID\"\n" +
                        "AND NOT (\n" +
                        "\"Subscription\".\"Period\" = \"PlanPeriod\".\"Period\"\n" +
                        "AND \"Subscription\".\"PeriodType\" = \"PlanPeriod\".\"PeriodType\"\n" +
                        ")\n" +
                        "AND \"PlanPeriod\".\"Enabled\" = 1\n" +
                        "AND \"PlanPeriod\".\"Trial\" = 0\n" +
                        "LEFT JOIN (\n" +
                        "SELECT\n" +
                        " \"DefPayTool\".\"PayToolID\",\n" +
                        " \"PayTool\".\"CutNumber\",\n" +
                        " \"PayTool\".\"PaySystem\",\n" +
                        " \"PayTool\".\"OwnerAccountID\"\n" +
                        "FROM\n" +
                        " \"PayTool\"\n" +
                        "INNER JOIN \"DefPayTool\" ON \"DefPayTool\".\"AccountID\" = \"PayTool\".\"OwnerAccountID\"\n" +
                        "WHERE\n" +
                        " \"PayTool\".\"IsSuspended\" = 0\n" +
                        "GROUP BY\n" +
                        " \"PayTool\".\"PayToolID\",\n" +
                        " \"PayTool\".\"CutNumber\",\n" +
                        " \"PayTool\".\"PaySystem\",\n" +
                        " \"PayTool\".\"OwnerAccountID\",\n" +
                        " \"DefPayTool\".\"PayToolID\"\n" +
                        "HAVING\n" +
                        " COUNT (*) > 0\n" +
                        ") AS \"ActivePaytool\" ON \"ActivePaytool\".\"OwnerAccountID\" = \"Account\".\"AccountID\"\n" +
                        "WHERE\n" +
                        "\"Account\".\"AccountID\" = ? " +
                        "AND \"Account\".\"TaxRegID\" = ? " +
                        "ORDER BY\n" +
                        "\"Account\".\"AccountID\",\n" +
                        "\"Subscription\".\"subscriptionID\";";

        return (ArrayList) this.pbaJdbc.query(sql, new Object[] { accountId, taxRegId }, bpm);
    }

Follows is the accountQueryResult class which is currently used as mapper.

package com.store.models.query;

public class accountQueryResult {
    private int AccountID;

    private int VendorAccountID;

    private String CompanyName;

    private String AdminPhAreaCode;

    private String AdminPhNumber;

    private String AdminFaxAreaCode;

    private String AdminFaxNumber;

    private String AdminEmail;

    private String PersPhAreaCode;

    private String PersPhNumber;

    private String PersFaxAreaCode;

    private String PersFaxNumber;

    private String PersEmail;

    private int TaxStatus;

    private String CutNumber;

    private String PaySystem;

    private int subscriptionID;

    private String SubscriptionName;

    private int Status;

    private int PlanID;

    private int resourceID;

    private String ResourceName;

    private int IncludedValue;

    private int Amount;

    private String IdParameter;

    private String Value;

    private int UsersID;

    private String LoginID;

    private String Address1;

    private String Address2;

    private String City;

    private String Zip;

    private int period;

    private int planPeriodID;

    private int periodType;

    private double renewalFee;

    private double setupFee;

// Ommited getter and setters
}

Please assist, how to change this one of the suggestions was using JPA, preferably with minimal changes as to avoid having to change all the business logic that processes the resulting array later..

Thanks!

Answer 1

The first thing when tackling a performance question like this is to find out where the time is spent.

Given your profilers information, it strikes me as odd that getting the ints seems to take so much longer than getting the other fields. I see following possible reasons:

1. the getInt does some expensive conversion, eg the JDDBC driver might return BigDecimal which then have to get converted to int .

2. it is just the first column to get accessed and actually triggers fetching and spends lot's of time waiting.

For the overall performance another problem might be the use of reflection by BeanPropertyRowMapper but this doesn't even show up in the profiler so far.

What I consider more likely is that the statement might return after 2 seconds but actually takes much longer to fetch all the results.

To clarify the situation I'd recommend testing the following:

1. Inspect the ResultSet returned by the JDBC driver for the datatypes it actually contains and check if you can get something more suitable out of it by changing column types or by casting in the SQL statement.

2. Obtain all the columns from all the rows without doing anything expensive with them. Just make sure the optimizer doesn't get rid of the access. For example you might want to create a hash from each value and add all hashes and print that at the end. Check the performance of that.