Why does my prime factorization code in processing 3 use so much memory and how can I make it use less memory?

Question

I wrote some code to create a list of the prime factorization of every number. The code works as expected for making lists of 1,000,000 or fewer elements but runs out of memory when I try to make larger lists. I was able to make a list of 1,000,000 elements at Processing's default memory limit of 256MB in about 100 seconds. Increasing the memory limit to 1024MB the code runs for about 45 minutes then runs out of memory when attempting to make a list of 2,000,000 elements. I believe the issue has something to do with memory management.

Here's the code.

int listLength=1000;
boolean failed=false;

void setup() {
  int time=millis(); //keep track of how long it takes to do things
  //get prime data
  String[] primesString=loadStrings("primes.txt");
  int[] primes=new int[primesString.length];
  for(int i=0; i<primes.length; i++) {
    primes[i]=int(primesString[i]);
  }
  println("loaded primes in "+((float(millis())-time)/1000)+" seconds");
  //do prime factorizations
  String[] list=new String[listLength];
  for(int i=0; i<listLength; i++) {
    int boi=i+1;
    list[i]=boi+"=";
    int primeIndex=0;
    while(boi!=1) {
      if(primeIndex==primes.length) { println("ERROR: not enough primes indexed"); boi=1; i=listLength; failed=true; }
      else {
        if(boi%primes[primeIndex]==0) {
          int count=1; boi/=primes[primeIndex];
          while(boi%primes[primeIndex]==0) {
            boi/=primes[primeIndex];
            count++;
          }
          list[i]+="p"+primeIndex+"^"+count+"*"; //list[i]+=primes[primeIndex]+"^"+count+"*";
        }
        primeIndex++;
      }
    }
    list[i]=list[i].substring(0, list[i].length()-1);
  }
  println("prime factored in "+((float(millis())-time)/1000)+" seconds");
  //save data
  if(!failed) {
    saveStrings(listLength+" prime factored.txt", list);
  }
  println("saved data in "+((float(millis())-time)/1000)+" seconds");
  //close program
  exit();
}

Answer 1

The major issue is, that you generate the entire output to a string list in memory ( list ) and write it to the file at the end. Write the file line by line. It is not necessary to keep the strings in memory. Use createWriter() and write each line by .println() to the file (see PrintWriter ).
Furthermore it is not necessary to keep the list of primesString in memory. In your algorithm just the integral prime numbers are needed. Move the code which reads the source data to a function, then the list of strings is just local in the scope of the function:

int listLength=1000;
boolean failed=false;

int[] GetPrimeData()
{
    String[] primesString=loadStrings("primes.txt");
    int[] primes=new int[primesString.length];
    for(int i=0; i<primes.length; i++) {
        primes[i]=int(primesString[i]);
    }
    return primes;
}

void setup() {
    int time=millis(); //keep track of how long it takes to do things

    //get prime data
    int[] primes = GetPrimeData();
    println("loaded primes in "+((float(millis())-time)/1000)+" seconds");

    PrintWriter output = createWriter(listLength + " prime factored.txt"); 

    //do prime factorizations
    for (int i=0; i<listLength; i++) {
        int boi=i+1;
        String newentry = boi+"=";
        int primeIndex=0;
        while (boi!=1) {
            if (primeIndex==primes.length) {
                println("ERROR: not enough primes indexed");
                boi=1; i=listLength; failed=true; 
            } else {
                if (boi%primes[primeIndex]==0) {
                    int count=1; boi/=primes[primeIndex];
                    while (boi%primes[primeIndex]==0) {
                        boi/=primes[primeIndex];
                        count++;
                    }
                    newentry += "p"+primeIndex+"^"+count+"*"; //list[i]+=primes[primeIndex]+"^"+count+"*";
                }
                primeIndex++;
            }
        }
        output.println(newentry.substring(0, newentry.length()-1));
    }

    output.close();
    println("prime factored in "+((float(millis())-time)/1000)+" seconds");

    //close program
    exit();
}