当向量很大时，如何获取R中向量的所有可能分区的列表？

Question

I am trying to use R to find all possible ways to partition the vector x of length n into at most m partitions . I know how to do then when n is small:

library(partitions)
x <- c(10, 20, 30, 40)
n <- length(x)
m <- 3

# In how many ways can we partition n objects into at most m patitions
parts <- restrictedparts(n, m)
sets <- setparts(parts)

In this example the value of sets is:

[1,] 1 1 1 1 2 1 1 1 1 1 1 2 2 2
[2,] 1 1 1 2 1 2 1 2 2 1 2 1 1 3
[3,] 1 2 1 1 1 2 2 1 3 2 1 3 1 1
[4,] 1 1 2 1 1 1 2 2 1 3 3 1 3 1

Each columns of sets tells me, for each unique arrangement, into which partition each item in x should be allocated.

The problem occurs when n is large:

n <- 15
m <- 4
parts <- restrictedparts(n, m)
# This expression will max out your CPU usage and eventually run out of memory.
sets <- setparts(parts)

How can I do this operation without running out of memory? I doubt there's a fast way to do it, so can I suspect I'll have to do it in batches and write to the disk.

Answer 1

If like me you are not a superstar in combinatorics but you trust that partitions has it right, then at least you can make use of the package's code to compute the final number of partitions. Here I hacked the setparts function so, instead of the partitions themselves, it returns the number of partitions:

num.partitions <- function (x) {
    if (length(x) == 1) {
        if (x < 1) {
            stop("if single value, x must be >= 1")
        }
        else if (x == 1) {
            out <- 1
        }
        else return(Recall(parts(x)))
    }
    if (is.matrix(x)) {
        out <- sum(apply(x, 2, num.partitions))
    }
    else {
        x   <- sort(x[x > 0], decreasing = TRUE)
        out <- factorial(sum(x))/(prod(c(factorial(x), 
                                         factorial(table(x)))))
    }
    return(out)
}

Let's check that the function is returning the correct number of partitions:

num.partitions(restrictedparts(4, 3))
# [1] 14
ncol(setparts(restrictedparts(4, 3)))
# [1] 14

num.partitions(restrictedparts(8, 4))
# [1] 2795
ncol(setparts(restrictedparts(8, 4)))
# [1] 2795

Now let's look at your large case:

num.partitions(restrictedparts(15, 4))
# [1] 44747435

That is indeed quite a lot of partitions... Regardless of how well or not setparts is written, the output cannot fit in a single array:

sets <- matrix(1, 15, 44747435)
# Error in matrix(1, 15, 44747435) : 
#  cannot allocate vector of length 671211525

So yes, you'd have to write your own algorithm and store to a list of matrices, or if it is too much for your memory, write to a file if that's really what you want to do. Otherwise, given the rather large number of permutations and what it is you want to do with them, go back to the drawing board...

Answer 2

If you want to calculate them in batches, it appears this may be possible for at least some of the columns. I was not able to complete a calculation of several of the individual columns in restrictedparts(15,4) on a machine like yours. Up to column 40 I could get success in batches of 5-10 columns at a time but above that there were several single columns that did report a number of columns before throwing a malloc error. So you may simply need a bigger machine. On my Mac that has 32 GB constructing the 53rd column consumed half of the memory. The estimates for number of columns on the big machine agreed with the report on the 4GB machine:

> ncol( setparts( restrictedparts(15,4)[,53]))
[1] 6306300
R(317,0xa077a720) malloc: *** mmap(size=378380288) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

(I offer no opinion on whether this is a sensible project. )

Answer 3

As I could not install the partitions package (missing libraries), I came up with this :

 ## Recursive function to get all partitions of a vector 
 ## Returns a list of logical vectors
 parts <- function(x) { 
   if (length(x) == 1) return(list(FALSE, TRUE))
   do.call(c, lapply(parts(x[-1]), function(y) list(c(FALSE, y), c(TRUE, y))))
 }

This function takes a vector and returns a list of logical vectors of the same size. The number of vectors in the list is the number of possible partitions, (2^n). It can't handle huge n, but on my pc it runs n=19 in less than a second.

If you want only the non-empty partitions, and no duplicates, use :

 partitions <- parts(x)
 partitions <- partitions[1:(length(partitions)/2)][-1]