如何将多个arrayList替换为单个Collection Java

Question

I am writing this utility code which dumps excel sheet into database (postgres in this case). I used poi's HSSF technique to deal with the excel sheet. I am storing each column of excel sheet into separate ArrayLists of type String. In this case when the number of columns in .xls are greater than 23 eclipse is generating an error 'heap memory full'. My question is can I combine these arraylists into one collection object and which method should I use?

public ArrayList<String> getList(String path, String srnoStr,
        String nameStr, String dobStr, String genderStr, String addressStr,
        String pinStr, String mobStr, String eIdStr, String categoryStr,
        String branchStr) throws IOException, SQLException {

    ArrayList<String> errorList = new ArrayList<String>();
    ArrayList<String> cellError = null;

    // String error=null;
    // OrderedMap errorMap=new LinkedMap();
    // errorMap=null;

    List<Cell> cells_srno = new ArrayList<Cell>();
    List<Cell> cells_name = new ArrayList<Cell>();
    List<Cell> cells_dob = new ArrayList<Cell>();
    List<Cell> cells_gender = new ArrayList<Cell>();
    List<Cell> cells_address = new ArrayList<Cell>();
    List<Cell> cells_pin = new ArrayList<Cell>();
    List<Cell> cells_mob = new ArrayList<Cell>();
    List<Cell> cells_eId = new ArrayList<Cell>();
    List<Cell> cells_category = new ArrayList<Cell>();
    List<Cell> cells_branch = new ArrayList<Cell>();

            try {
        int srnoIndex = 0;
        srnoIndex = getIndex(srno, path);
        cellError = fillList(srnoIndex, srNoId, cells_srno, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int nameIndex = 0;
        nameIndex = getIndex(name, path);
        cellError = fillList(nameIndex, nameId, cells_name, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int dobIndex = 0;
        dobIndex = getIndex(dob, path);
        cellError = fillList(dobIndex, dobId, cells_dob, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int genderIndex = 0;
        genderIndex = getIndex(gender, path);
        cellError = fillList(genderIndex, genderId, cells_gender, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int addressIndex = 0;
        addressIndex = getIndex(address, path);
        cellError = fillList(addressIndex, addressId, cells_address, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int pinIndex = 0;
        pinIndex = getIndex(pin, path);
        cellError = fillList(pinIndex, mobId, cells_pin, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int mobIndex = 0;
        mobIndex = getIndex(mob, path);
        cellError = fillList(mobIndex, pinId, cells_mob, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int eIdIndex = 0;
        eIdIndex = getIndex(eId, path);
        cellError = fillList(eIdIndex, eIdId, cells_eId, path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int categoryIndex = 0;
        categoryIndex = getIndex(category, path);
        cellError = fillList(categoryIndex, categoryId, cells_category,
                path);
        if (cellError != null)
            errorList.addAll(cellError);

        cellError = null;
        int branchIndex = 0;
        branchIndex = getIndex(branch, path);
        cellError = fillList(branchIndex, branchId, cells_branch, path);
        if (cellError != null)
            errorList.addAll(cellError);

        // System.out.println(cells);
        int n = cells_srno.size();
        int k = 0;
        System.out.println("Total number Rows = " + (n - 1));
        int j = 0;
        // System.out.println("Column name=" +cells.get(0));
        if (errorList.isEmpty()) {
            for (int i = 1; i < n; i++) {
                k = Insert(cells_srno.get(i), cells_name.get(i),
                        cells_dob.get(i), cells_gender.get(i),
                        cells_address.get(i), cells_pin.get(i),
                        cells_mob.get(i), cells_eId.get(i),
                        cells_category.get(i), cells_branch.get(i));
                if (k > 0)
                    j++;

            }

        } else {
            System.out.println("Error");
        }

        /*
         * System.out
         * .println("Total no. of physical row in you Excel Sheet=" + j);
         * System.out
         * .println("Operation Successfull!! kindly check the database!!! "
         * );
         */

    } catch (Exception e) {
        e.printStackTrace();
    }
    return errorList;
}

Answer 1

If memory shortage is the problem, I don't think your approach is very good. You should reduce your memory footprint by streaming, rather than by having one big collection instead of several collections. You are only going to save very little that way, and as soon as your input data grows a bit bigger you'll have the same issue again.

Ie process (insert) one row at a time, instead of "fill"ing the list upfront. If are using a sane library to read the input excel sheet which also does streaming, this should make it possible to handle large amounts of data practically as long as it fits on disk.

Answer 2

I would go with a Map and an enum as key and refactor like this:

public List<String> getList(final String path, final Map<CellType, String> data) {
    final Map<CellType, List<Cell>> cells = new EnumMap<CellType, List<Cell>>();
    for (final CellType cellType : CellType.values()) {
        cells.put(cellType, new ArrayList<Cell>());
    }

    final List<String> errorList = new ArrayList<String>();
    try {
        for (final CellType cellType : data.keySet()) {
            final int index = getIndex(data.get(cellType), path);
            final List<String> cellError = fillList(index, /* the ID for cellType */, cells.get(cellType), path);
            if (cellError != null) {
                errorList.addAll(cellError);
            }
        }

        // ... (use the map)
    } catch (final Exception e) {
        e.printStackTrace();
    }

    return errorList;
}

public static enum CellType {
    SRNO, NAME, DOB // ...
}

(not tested)

NB: srNoId , nameId etc are not defined in the code you posted so I don't know what they are