Cannot stack 2D arrays into 3D array

Question

I am trying to stack some 2D arrays of the same size my code looks like this:

import numpy as np
a = np.random.randn(30_140_100)
b = np.random.randn(30_140_100)
c = np.random.randn(30_140_100)
d = np.random.randn(30_140_100)
e = np.random.randn(30_140_100)
f = np.random.randn(30_140_100)

array = np.vstack((a,b,c)).reshape(3, 5490, 5490)

I am using a old computer to do so (32 Bits). I heard that one can process the array in chunks and store the results in hard disk. I don't know if that would be a solution, and don't know the actual implementation in python code.

Thank you very much!

Best Regards

Answer 1

A memory error means that python has run out of memory to store its data. A 32bit machine can only support a maxmimum of 4 Gb of memory, which isn't a lot given the size of the random array in your example. Unfortunately the only way around this is to increase the memory on the system, which can't be done past 4Gb on a 32bit computer.