代表给定范围内所有数字的最佳方法是什么？ （一些限制）

Question

Background: Creating a game similar to Mastermind but on steroids. Currently trying to optimize AI and also increase my limits. Let me explain:

For my AI algorithm I need to produce some sort of data structure that can represent all possible numbers given 0 - a limit. I also need to remove numbers from such structure to reflect possible numbers.

Right now in my game I use List to represent a number per se. That way I can resemble any base and depending on events in game I can add a digit to such number. I know it isn't the best way to go about this but regarding game only the AI is where all the heavy work is being done.

I created a way to have an ArrayList> but the problem is when the limit becomes bigger than Integer.MAX_VALUE I run into a lot of problems with causing the program to freeze and such and also leads to huge memory issues. I pulled some code showing how I am currently generating all sequences. I also just implemented shitty job of multithreading so you guys don't have to wait long when running the code:

public class Recursive {

    private static List<List<Integer>> allSeqL = new ArrayList<List<Integer>>();
    private static final Object addPossibleSequenceLock = new Object();
    public static void main(String[] args){

        //Create Digits
        List<Integer> digits = new ArrayList<Integer>();
        int index = 0;
        while(10 > index){
            digits.add(index);
            index++;
        }
        System.out.println(digits.size()+"SIZE OF DIGITS");

        //Get Amount of Threads to run at once
        int recommendedAmountWT = Runtime.getRuntime().availableProcessors()*3/4;

        //Create Working List
        List<Thread> workTL = new ArrayList<Thread>();
        for(Integer number : digits){
            Thread t = new Thread(new Runnable(){
                private int number;
                private List<Integer> digits;
                public void run(){
                    int sizeOfSequence = 10;
                    boolean dupAllowed = true;
                    Integer[] sequence = new Integer[sizeOfSequence];
                    sequence[0] = number;
                    if(dupAllowed){
                        Recursive.recursiveAllPossible(new ArrayList<Integer>(digits), sequence, 1, dupAllowed);
                    }   else{
                        digits.remove(new Integer(number));
                        Recursive.recursiveAllPossible(digits, sequence, 1, dupAllowed);
                    }
                }
                public Runnable init(int number, List<Integer> digits){
                    this.number = number;
                    this.digits = digits;
                    return this;
                }
            }.init(number, new ArrayList<Integer>(digits)));
            workTL.add(t);
        }


        System.out.println(workTL.size()+"SIZE WORKTL");

        //Run only Recommended amount of work threads at once
        List<Thread> tempTL = new ArrayList<Thread>();
        while(workTL.size() != 0){
            int startThread = 0;
            while(workTL.size() != 0 && recommendedAmountWT > startThread){
                Thread t1 = workTL.get(0);
                tempTL.add(t1);
                workTL.remove(t1);
                startThread++;
                t1.start();
            }
            System.out.println(startThread);
            System.out.println(tempTL.size());
            System.out.println("##########################");
            for(Thread t : tempTL){
                try {
                    t.join();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            tempTL.clear();
        }
        System.out.println("TOTAL ALLSEQL: "+Recursive.allSeqL.size());
    }


    private static void recursiveAllPossible(List<Integer> digits, Integer[] sequence, int index, boolean dupAllowed){
        for(Integer number : digits){
            sequence[index] = number;
            if(sequence.length-1 > index){
                List<Integer> newPositionL = new ArrayList<Integer>(digits);
                if(!dupAllowed){
                    newPositionL.remove(number);
                }
                Recursive.recursiveAllPossible(newPositionL, sequence, new Integer(index+1), dupAllowed);
            }   else{
                Recursive.addToL(Arrays.asList(sequence));
            }
        }
    }

    /**
     * Method : Add to all sequence List
     * @param sequence : Sequence being added
     * Reason this has a lock is because when creating this list there are multiple threads
     */
    public static void addToL(List<Integer> sequence){
        synchronized(Recursive.addPossibleSequenceLock){
            Recursive.allSeqL.add(sequence);
            System.out.println(sequence);
        }
    }
}

Basically this works by setting number in digits to index of sequence array, cycles until it can't set any more, then it adds it to the all sequence list. Then it loops back to the previous and sees if there is another digit to add. Adds and repeats.

The AI really doesn't have to follow List format compared to the rest of the game. However, to compare numbers I would have to convert whatever to List or rewrite my Result class but I would rather not.

So any tips of what I should do to solve this problem would be appreciated. My solutions I came up with is implement some sort of Tree. To be honest though I don't know much about Trees and don't feel to confident regarding this. Another solution was to create a new ArrayList when it reaches the Integer.MAX_VALUE with new sequences but that just seems like I'll be sweeping the problem under the rug.

Third solution was use LinkedList but still shits out so to speak (I have tried this one).

Answer 1

Given the size of the problem, have you considered using Sparse Matrixes?

I think maybe you can represent a disjoint list of non-consecutive numbers as simply a list of ranges:

my_ranges = [(0,10),(12,1500),(1502,25000000)]

and this way, when you need to remove a number you can (some pythonic pseudocode):

insert():
for subrange in my_ranges:
   if 27 in my_ranges:
       # do nothing
   else:
       # build intermediate ranges with 27 included
return new_ranges

delete():
for subrange in my_ranges:
   if 27 in my_ranges:
       # split this range into two ranges, before and after 27
return new_ranges


exists():
for subrange in my_ranges:
   if 27 in my_ranges:
       return 1
return return 0

This way you can solve the "representing a lot of numbers, with some exceptions, without actually running out of memory.

alternatively, you can use sparse matrixes :).

Answer 2

I once made a collection class that might be of interest to your problem.

It acts as Set<Long> , but it uses a very efficient internal representation with a good space/time tradeoff.

The key idea is to store consecutive ranges, more preceisely, the numbers where the function "contains" changes value. In the worst case the size is proportional to number of elements, which happens when elements represent a checkerboard (1, 3, 5, 7, 9 etc.), but even in that case, it's still just N. Empty and full set have constant size 1. Range 10000-30000 has a size of 2 instead of 20000, etc. Otherwise, ranges are merged and splitted dynamically when adding/removing values.

The class supports standard set operations such as intesection, union etc. In my experience it has worked great, but I probably didn't have as much requirements as you do. In practice, the number of "delta points" is still limited by java's Integer.MAX_VALUE , but it's reasonably straightforward to extend it to Long.MAX_VALUE or even BigInteger .

To address your problem, you should observe the patterns in your data. Is the pattern random or not? Are the values localized? Etc. Different usages call for different data structures and algorithms with different trade-offs. If you're just starting, take a look at HashSet , TreeSet or collections in Google Guava project to get you going, then look for the patterns in your data. There are also some probabilistic structures such as Bloom filters that you might find useful. Ultimately, the patterns you observe in your actual data will give you the best answer.