Excel VBA - populating Variant array increase memory usage

Question

I'm new here and sorry in advance if my question was answered already, but I am searching for several days now and found no solution or explanation yet.

The problem is that during population of a variant array memory consumption constantly rises. The array is dimensioned, and when it is first dimensioned, I can see how memory consumption increases by a corresponding amount. I think a small piece of code will explain it better than words:

Sub test()
Dim Arr(1 To 1000000, 1 To 10) As Variant
Dim i As Long, j As Integer

For i = 1 To 1000000
    For j = 1 To 10
        Arr(i, j) = "AAAAAAAAA"
    Next j

Next i

End Sub

My experience with this code is - after array dimensioning memory cons. increases by approx. 160 MB (which is 10*1 000 000 * 16). After the execution has finished I have and additional increase by about 400 MB! Interestingly - if you replace the string AAAA by a number, this problem vanishes.

In the actual program (this is just a test code) I have a bigger array of approx. 600 MB, but at the end of code execution memory usage is 3.4 GB! As a consequence an "Out of memory" error.

Can anyone please explain this?

Thanks in advance, Den

Answer 1

What you are seeing is caused by the fact that when a variant contains a string, the actual string data is not stored internally. A variant consists of 16 bytes. The layout of those 16 bytes is described here . Note that 8 of those bytes contain things like type information and the other 8 contain the actual data when that data is numerical . When the data is a string, those 8 bytes are not enough. Instead, those 8 bytes contains a pointer to a BSTR structure which consists of 4 bytes holding the length of a string, the actual characters in the string (with 2 bytes per character) and a 2-character null terminator. For your string of length 9, that works out to 24 bytes, hence a total of 16 + 24 = 40 bytes for storing that string (and the variant that it points to). This explains why it takes at least 400 MB to store those 10 million strings when you store them that way.

As you can see, variants are a memory hog. If you really need to use Excel VBA for this, you should avoid arrays of variants like a plague. Strings themselves are not very efficient. Perhaps you can do something like store the ascii codes for the characters as bytes or combine many strings into a single string which can be split when needed.