带有递归的字符串排列无法完成超过9个字符的处理-抛出OutOfMemoryError

Question

I have a recursion method that finds all possible combinations of the characters in a given String . The recursion method works perfectly with any String which contains 9 letters or less. It completes the recursion method with 9 letters in around 4 seconds. However, once there is more than 9 letters it runs into problems. The method runs for about 2 minutes with numerous GC lines being written to the log, when the process finally completes I get a Throwing OutOfMemoryError exception. Is any string over 9 letters asking to much? This is all done on an AsyncTask .

Here is the stacktrace:

07-29 12:24:39.335 17389-17389/com.example.test.apptest W/art: Throwing OutOfMemoryError "Failed to allocate a 76 byte allocation with 4194304 free bytes and 12MB until OOM; failed due to fragmentation (required continguous free 4096 bytes for a new buffer where largest contiguous free 0 bytes)" (recursive case)
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art: "main" prio=5 tid=1 Runnable
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   | group="main" sCount=0 dsCount=0 obj=0x73e1e258 self=0xb40f4500
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   | sysTid=17389 nice=0 cgrp=default sched=0/0 handle=0xb77e1c00
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   | state=R schedstat=( 0 0 0 ) utm=899 stm=114 core=1 HZ=100
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   | stack=0xbf326000-0xbf328000 stackSize=8MB
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   | held mutexes= "mutator lock"(shared held)
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   native: #00 pc 0058bd02  /system/lib/libart.so (art::DumpNativeStack(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, int, char const*, art::ArtMethod*, void*)+226)
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   native: #01 pc 0055285e  /system/lib/libart.so (art::Thread::ThrowOutOfMemoryError(char const*)+542)
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   native: #02 pc 0029b6cd  /system/lib/libart.so (art::gc::Heap::ThrowOutOfMemoryError(art::Thread*, unsigned int, art::gc::AllocatorType)+1559)
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   native: #03 pc 002a4a62  /system/lib/libart.so (art::gc::Heap::AllocateInternalWithGc(art::Thread*, art::gc::AllocatorType, unsigned int, unsigned int*, unsigned int*, unsigned int*, art::mirror::Class**)+5218)
07-29 12:24:39.344 17389-17389/com.example.test.apptest W/art:   native: #04 pc 001ade47  /system/lib/libart.so (art::mirror::PrimitiveArray<int>::Alloc(art::Thread*, unsigned int)+2423)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:   native: #05 pc 0054dd6e  /system/lib/libart.so (_jobject* art::Thread::CreateInternalStackTrace<false>(art::ScopedObjectAccessAlreadyRunnable const&) const+298)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:   native: #06 pc 0047fc31  /system/lib/libart.so (art::Throwable_nativeFillInStackTrace(_JNIEnv*, _jclass*)+52)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:   native: #07 pc 0002475e  /data/dalvik-cache/x86/system@framework@boot.oat (Java_java_lang_Throwable_nativeFillInStackTrace__+98)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.Throwable.nativeFillInStackTrace!(Native method)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.Throwable.fillInStackTrace(Throwable.java:166)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.Throwable.<init>(Throwable.java:95)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.Error.<init>(Error.java:48)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:46)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:44)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.AbstractStringBuilder.enlargeBuffer(AbstractStringBuilder.java:95)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.AbstractStringBuilder.append0(AbstractStringBuilder.java:146)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.StringBuilder.append(StringBuilder.java:216)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at com.android.internal.os.RuntimeInit.Clog_e(RuntimeInit.java:61)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at com.android.internal.os.RuntimeInit.-wrap0(RuntimeInit.java:-1)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at com.android.internal.os.RuntimeInit$UncaughtHandler.uncaughtException(RuntimeInit.java:94)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:693)
07-29 12:24:39.345 17389-17389/com.example.test.apptest W/art:     at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:690)
07-29 12:24:39.346 17389-17389/com.example.test.apptest I/Process: Sending signal. PID: 17389 SIG: 9

and the recursion method:

//Initial permutation method
public void permutation(String lettersToCombine) {
  permutation("", lettersToCombine);
}

//Recursion method to find all combinations of letters in a given string.
private void permutation(String prefix, String passedLetters) {

    //Set int to the size of the String passed.
    int lengthOfPassedLetters = passedLetters.length();
    //Add the prefix to the ArrayList. 
    if (lengthOfPassedLetters == 0) myList.add(prefix);
    //Loop through this the amount of times of the size of passed letters.
    for (int i = 0; i < lengthOfPassedLetters; i++) {
        /*Call this same method every time the loop is entered. Setting prefix to the character at position of i
        prefix is passed into the method for first argument. For the second argument another String is passed containing
        the second argument made up of the letters already processed and the letters left too.
        */
        permutation(prefix + passedLetters.charAt(i), passedLetters.substring(0, i) + passedLetters.substring(i + 1,
                lengthOfPassedLetters));
    }

Answer 1

The real problem behind is that the String objects didn't disponse until the method finish. And, of course, this is recursive so it doesn't end until the last method call ends.

1. I suggest to use StringBuilder.
2. ¿Maybe with arrays?
3. You can try setting your local variables to null.
4. I suggest you to use Loops.

If you still can't solve it, I will try to help with some code.

package pruebas;

import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.function.Consumer;

/**
 *
 * @author Oscar
 */
public class Permutations {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {

        new PermutatedResult("ABCDEFGHIJKLMNÑOPQRSTUVWXYZ").printPermutations();
    }

}

class PermutatedResult {

    private String input;

    public PermutatedResult(String input) {
        this.input = input;
    }

    public void printPermutations() {

        deferredProcess(s -> System.out.println(s));
    }

    public String[] getPermutations() {
        ArrayList<String> list = new ArrayList<String>((int)Math.pow(2, input.length()));
        deferredProcess(s -> list.add(s));

        return list.toArray(new String[input.length()]);
    }

    public void deferredProcess(Consumer<String> actionForEverySolution) {

        int length = input.length();
        long combinations = (long) Math.pow(2, length);

        StringBuilder combination = new StringBuilder(length);
        for (long i = 0; i < combinations; i++) {

            for (int j = 0; j < length; j++) {

                if (((i >> j) & 1) == 1) {
                    combination.append(input.charAt(j));
                }
            }
            actionForEverySolution.accept(combination.toString());
            combination.setLength(length);
        }
    }
}

Here's some code, tomorrow I will optimize/refactor and will explain it. It works for 64 length Strings, but it take a will since is a bruteforce algorithm.

Edit:

Some code to work with near ant length (Integer.MAX_VALUE) and is so fast (2**27 combinations in 14 seconds). In the code, I use an iterator in order to not store all the combinations and save RAM. So you have to iterate through the PermutatedResult to get the values and use them. If you want more than that length, It's possible, but instead of having one byte[] we need a byte[][]. If the algorithm still takes to much, I can try to use multithreads to speed up and optimize it a bit more.

package pruebas;

import java.util.Iterator;

public class Permutations {

    public static void main(String[] args) {

        PermutatedResult result = new PermutatedResult("ABCDEFGHIJKLMNÑOPQRSTUVWXYZ");

        int combinaciones = 0;
        while (result.hasNext()) {
            combinaciones++;
            result.next(); // This line or the next
            //System.out.println(result.next());
        }
        System.out.println(combinaciones);
    }
}

class PermutatedResult implements Iterator<String> {

    private char[] input;
    private Boolean next;
    private byte[] lastCombination;
    private StringBuilder combination;

    public PermutatedResult(String input) {

        /* Some checks, but we need more */
        if (input == null || input.length() == 0) {
            this.next = false;
            return;
        }

        double posibleCombinations = input.length();

        /* Max length of an array... */
        if (posibleCombinations < Integer.MAX_VALUE) {

            this.input = input.toCharArray();
            this.lastCombination = new byte[(int) posibleCombinations];
            this.combination = new StringBuilder(this.input.length);
            this.next = true;
            this.nextCombination();
        }
    }

    @Override
    public boolean hasNext() {
        return this.next;
    }

    @Override
    public String next() {

        if (!next)
            return null;

        combination.setLength(0);

        for (int i = 0; i < input.length; i++) {

            if (lastCombination[i] == 1) /* If we have to use the letter at this position... */
                combination.append(input[i]);
        }

        this.nextCombination();
        return combination.toString();
    }

    private void nextCombination() {

        int remainder = 1;
        int length = lastCombination.length;
        int sum;

        /* Sum 1 to the bits -> 1111 + 1 = 10000 */
        for (int i = 0; remainder == 1 && i < length; i++) {
            sum = ++lastCombination[i];
            remainder = sum >> 1; // This will got the remainder -> 1 + 1 = 10 so shifting (10 >> 1) we got 1 as remainder.
            lastCombination[i] = (byte) (sum & 1); // With this we only take the last bit so 1 + 1 = 10 -> 10 & 1 = 0
        }

        next = remainder != 1; // If there is some remainder we end all the array and finish
    }
}

Answer 2

Your code is very ineffecient in terms of memory. Permutation of 10 letters is 10! ~ 3.5 millions - this gives you an estimation of String objects that are created and destroyed all the time which leads to the memory fragmentation.

failed due to fragmentation (required continguous free 4096 bytes for a new buffer where largest contiguous free 0 bytes)

Answer 3

To calculate permutation of string we need n! callstack where n is length of the string. There is limit for callstacks. Thats why Geek warns you before using recursion.