Use List.subList to perform element batching logic

Question

I need to take a ArrayList that has a few thousand elements and process them a 1000 elements at a time. So I need to take a large ArrayList (lets say it has 3500 elements) and create new ArrayList with 1000 max elements. I'm using the below code, but I feel its not that efficient, any suggestions?

import java.util.ArrayList;
import java.util.List;

public class SubList {

  public static void main(String[] args) {
    List<Long> ids = new ArrayList<Long>();
   for(int i=0; i<3500; i++){
     ids.add(Long.valueOf(i));
   }
   int threashold=1000;
   double iteration = (double)ids.size()/threashold;
   int fromIndex=0;
   int toIndex=threashold;
   for(int i=0; i<iteration; i++){
     if(i + 1 >= iteration){
       toIndex = ids.size();
       List<Long> list = ids.subList(fromIndex, toIndex);
       processList(list);
     }else{
       List<Long> list = ids.subList(fromIndex, toIndex);
       processList(list);
       fromIndex += threashold;
       toIndex += threashold;
     }
   }

  }
  private static void processList(List<Long> ids){
    System.out.println("size=" + ids.size());
  }

}

Answer 1

I checked the implementation of the method sublist in the class ArrayList (form openjdk).

public List<E> More ...subList(int fromIndex, int toIndex) {
    subListRangeCheck(fromIndex, toIndex, size);
    return new SubList(this, 0, fromIndex, toIndex);
}

The returned class is not an instance of ArrayList but an instance of an inner class called SubList.

SubList(AbstractList<E> parent,
    int offset, int fromIndex, int toIndex) {
    this.parent = parent;
    this.parentOffset = fromIndex;
    this.offset = offset + fromIndex;
    this.size = toIndex - fromIndex;
    this.modCount = ArrayList.this.modCount;
}

SubList stores the relation with the original ArrayList and uses it instead of making a simple copy, so it should be as efficient as the original ArrayList, and the sublist is created in constant time.

So, your implementation may be ok.

Answer 2

I do something similar with String lists. I have converted it to Long for you. Its basically a list copy, except when it reaches the threshold, it stores the list and starts a new one. It doesn't process the lists inline like yours. But this is a 'neat' approach..

public static List<List<Long>> getSubLists(List<Long> list, int size) {
    List<List<Long>> outer = new ArrayList<List<Long>>();

    List<Long> subList = new ArrayList<Long>(size);
    for (Long l : list) {
        subList.add(l);

        if (subList.size() >= size) {
            outer.add(subList);
            subList = new ArrayList<Long>(size);
        }
    }
    if (subList.size() > 0) {
        outer.add(subList);
    }
    return outer;
}