How can I solve huge performance issues when comparing and updating data with java?

Question

I want to compare two lists with objects of the same type wich have about 30 Values. When there are differences the new Value should be updated to the database.
The part where my code gets really really slow, like one update every minute, is when the data should be updated.

reading all the datasets from db and excel is fast. But when it comes to the below code it already starts to nearly stop after 200 datasets.

I cant provide a complete code for you since the method getLists() calls a lot of functions wich would be far too much code in my opinion. But maybe you can still help me with my problem since the relevant part where the performance gets low occures at the given code.

Maybe somehow a lot of objects getting created or something like that?

EDIT: I run the program with IntelliJ. When I close the running application and start it again it loads the already slowly loaded datasets from the previous time really fast but then slows down again. Maybe something with the cache is wrong?

Here is my code:

public class MainWindow_Controller {

List<Pensioner> pensionerDB = new ArrayList();
List<Pensioner> pensionerExcel = new ArrayList();

Boolean checkStatusChange = false;
String pathDB = "";

@FXML
private TextField tfDBPath;

/**
 * This method compares the pensioners from database with the ones from the excel head file.
 * Based on matching pensionInsuranceNumbers the method checks if the status is different.
 * If it is different, the status gets updated.
 *
 * @param event
 */
@FXML
void getStatusChange(ActionEvent event) throws FileNotFoundException, IllegalAccessException, SQLException {
    String type = "Statuswechsel";
    if (pensionerDB.isEmpty() || pensionerExcel.isEmpty()) {
        getLists();
    }

    pathDB = tfDBPath.getText();
    //load pensioners from database in first list
    pensionerDB = array[0];

    //load pensioners from excel head file in second list
    pensionerExcel = array[1];

    //compare pensionInsuranceNumber from first list with pensionInsuranceNumber from second list and move missing datasets to third list
    List<Pensioner> result = new ArrayList();

    Database.connect(pathDB);

    //iterates over the two lists with pensioners in database and pensioners in excel files and adds the dataset with matching pensionInsuranceNumbers into the result list.
    for (int i = 0; i < pensionerExcel.size(); i++) {
        System.out.println("Processing Excelrow Number: " + i);
        for (int j = 0; j < pensionerDB.size(); j++) {
            updatePensionerData(pensionerExcel, pensionerDB, pathDB, i, j);
            if (pensionerExcel.get(i).getPensionInsuranceNumber() == pensionerDB.get(j).getPensionInsuranceNumber() && pensionerExcel.get(i).getStatusOld() != pensionerDB.get(j).getStatusOld()) {
                checkStatusChange = true;
                pensionerDB.get(j).setStatusNew(pensionerDB.get(j).getStatusOld());
                pensionerDB.get(j).setStatusOld(pensionerExcel.get(i).getStatusOld());
                result.add(pensionerDB.get(j));
                break;
            } else if (pensionerExcel.get(i).getPensionInsuranceNumber() == pensionerDB.get(j).getPensionInsuranceNumber() && pensionerExcel.get(i).getStatusOld() == pensionerDB.get(j).getStatusOld()) {
                break;
            }
        }
    }
    Database.close();

}

public void updatePensionerData(List<Pensioner> pensionerExcel, List<Pensioner> pensionerDB, String pathDB, int i, int j) {
    if (pensionerExcel.get(i).getPensionInsuranceNumber() == pensionerDB.get(j).getPensionInsuranceNumber()) {
        if (pensionerExcel.get(i).getIdkz() != pensionerDB.get(j).getIdkz()) {
            Database.updateIdkz(pensionerExcel.get(i), pathDB);
        }
        if (!pensionerExcel.get(i).getCompany().equals(pensionerDB.get(j).getCompany())) {
            Database.updateCompany(pensionerExcel.get(i), pathDB);
        }
        if (pensionerExcel.get(i).getPersonelId() != pensionerDB.get(j).getPersonelId()) {
            Database.updatePersonelId(pensionerExcel.get(i), pathDB);
        }

        if (!pensionerExcel.get(i).getBirthDate().isEqual(pensionerDB.get(j).getBirthDate())) {
            Database.updateBirthDate(pensionerExcel.get(i), pathDB);
        }
        //...
        //26 more if statements for the other values
        //...
    }
}

}

My database methods:

public static void updateIdkz(Pensioner p, String pathDB) {
    String update = String.format("UPDATE Pensionär SET idkz = ?");
    try {
        ps = connection.prepareStatement(update);
        ps.setInt(1, p.getIdkz());
        ps.executeUpdate();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

public static void updatePersonelId(Pensioner p, String pathDB) {
    String update = String.format("UPDATE Pensionär SET PersNr = ? Where pknr = ?");
    try {
        ps = connection.prepareStatement(update);
        ps.setInt(1, p.getPersonelId());
        ps.setInt(2, p.getPensionInsuranceNumber());
        ps.executeUpdate();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

public static void updateBirthDate(Pensioner p, String pathDB) {
    String update = String.format("UPDATE Pensionär SET Geburtsdatum = ? Where pknr = ?");
    try {
        ps = connection.prepareStatement(update);
        ps.setDate(1, Date.valueOf(p.getBirthDate()));
        ps.setInt(2, p.getPensionInsuranceNumber());
        ps.executeUpdate();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Thank you for your help

Answer 1

Something like this will be more efficient. You can re-use the PreparedStatements and even batch the updates to make the transactions faster. Also, as others have said, remember to close your PreparedStatements.

class Test implements Closeable {

    // reusable PreparedStatements
    private final PreparedStatement updateIdkz;
    private final PreparedStatement updatePersonelId;
    private final PreparedStatement updateBirthDate;

    public Test(Connection con) throws SQLException {
        this.updateIdkz = con
            .prepareStatement("UPDATE Pensionär SET idkz = ?");
        this.updatePersonelId = con
            .prepareStatement("UPDATE Pensionär SET PersNr = ? Where pknr = ?");
        this.updateBirthDate = con
            .prepareStatement("UPDATE Pensionär SET Geburtsdatum = ? Where pknr = ?");
    }

    /** Closes all the prepared statements. */
    @Override
    public void close() throws IOException {
        try {
            updateIdkz.close();
        } catch (SQLException e) {
        }
        try {
            updatePersonelId.close();
        } catch (SQLException e) {
        }
        try {
            updateBirthDate.close();
        } catch (SQLException e) {
        }
    }

    public void addBatchUpdateIdkz(Pensioner p) {
        try {
            updateIdkz.clearParameters();
            updateIdkz.setInt(1, p.getIdkz());
            updateIdkz.addBatch();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void addBatchUpdatePersonelId(Pensioner p) {
        try {
            updatePersonelId.clearParameters();
            updatePersonelId.setInt(1, p.getPersonelId());
            updatePersonelId.setInt(2, p.getPensionInsuranceNumber());
            updatePersonelId.addBatch();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void addBatchUpdateBirthDate(Pensioner p) {
        try {
            updateBirthDate.clearParameters();
            updateBirthDate.setDate(1, Date.valueOf(p.getBirthDate()));
            updateBirthDate.setInt(2, p.getPensionInsuranceNumber());
            updateBirthDate.addBatch();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void executeBatchUpdateIdkz() throws SQLException {
        updateIdkz.executeBatch();
    }

    public void executeBatchUpdatePersonelId() throws SQLException {
        updatePersonelId.executeBatch();
    }

    public void executeBatchUpdateBirthDate() throws SQLException {
        updateBirthDate.executeBatch();
    }

    public static void main(String[] args) {

        Pensioner p1 = null, p2 = null, p3 = null, p4 = null;
        Connection con = null;

        // try with resources to ensure you close the prepared statements
        try (Test t = new Test(con);) {
            // Update multiple Idkz
            t.addBatchUpdateIdkz(p1);
            t.addBatchUpdateIdkz(p2);
            t.addBatchUpdateIdkz(p3);
            t.addBatchUpdateIdkz(p4);
            t.executeBatchUpdateIdkz();

            // Update multile PersonelId
            t.addBatchUpdatePersonelId(p1);
            t.addBatchUpdatePersonelId(p2);
            t.addBatchUpdatePersonelId(p3);
            t.addBatchUpdatePersonelId(p4);
            t.executeBatchUpdatePersonelId();

            // etc...
        } catch (IOException e) {
            e.printStackTrace();
        } catch (SQLException e1) {
            e1.printStackTrace();
        }
    }
}

Answer 2

You can add a finally block for each try... catch and close the PreparedStatement object.

Example:

public static void updateIdkz(Pensioner p, String pathDB) {
  String update = String.format("UPDATE Pensionär SET idkz = ?");
  try {
      ps = connection.prepareStatement(update);
      ps.setInt(1, p.getIdkz());
      ps.executeUpdate();
  } catch (Exception e) {
      e.printStackTrace();
  }
  finally {
    ps.close();
  }

Let me know if it works.