在大范围内解析到阵列VBA的最有效方法

Question

I have a large range of data in excel that I would like to parse into an array for a user defined function. The range is 2250 x 2250. It takes far too long to parse each cell in via a for loop, and it is too large to be assigned to an array via this method:

dim myArr as Variant
myArr = range("myrange")

Just brainstorming here, would it be more efficient to parse in each column and join the arrays? Any ideas?

Thanks

Answer 1

You're nearly there.

The code you need is:

Dim myArr as Variant
myArr = range("myrange").Value2

Note that I'm using the .Value2 property of the range, not just 'Value', which reads formats and locale settings, and will probably mangle any dates

Note, also, that I haven't bothered to Redim and specify the dimensions of the array: the Value and Value2 properties are a 2-dimensional array, (1 to Rowcount, 1 to Col Count)... Unless it's a single cell, which will be a scalar variant which breaks any downstream code that expected an array. But that's not your problem with a known 2250 x 2250 range.

If you reverse the operation, and write an array back to a range, you will need to set the size of the receiving range exactly to the dimensions of the array. Again, not your problem with the question you asked: but the two operations generally go together.

The general principle is that each 'hit' to the worksheet takes about a twentieth of a second - some machines are much faster, but they all have bad days - and the 'hit' or reading a single cell to a variable is almost exactly the same as reading a seven-million-cell range into a variant array. Both are several million times faster than reading that range in one cell at a time.

Either way, you may as well count any operation in VBA as happening in zero time once you've done the 'read-in' and stopped interacting with the worksheet.

The numbers are all very rough-and-ready, but the general principles will hold, right up until the moment you start allocating arrays that won't fit in the working memory and, again, that's not your problem today.

Remember to Erase the array variant when you've finished, rather than relying on it going out of scope: that'll make a difference, with a range this size.

Answer 2

This works fine.

Sub T()
    Dim A() As Variant

    A = Range("A2").Resize(2250, 2250).Value2

    Dim i As Long, j As Long
    For i = 1 To 2250
        For j = 1 To 2250
            If i = j Then A(i, j) = 1
        Next j
    Next i

    Range("A2").Resize(2250, 2250).Value2 = A
End Sub

Answer 3

I think the best options are:

1. Try to limit the data to a reasonable number, say 1,000,000 values at a time.
2. Add some error handling to catch the Out of Memory error and then try again, but cut the size in half, then by a third, a quarter, etc...until it works.

Either way, if we're using data sets in the order of 5,000,000 values and you want to make sure that the program will run, you will need to adjust the code to chop up the data.