How to assign column values in large numpy array?

Question

I am new to Python programming and I have a problem in assigning specific values to the first column of a very large numpy.array.

This is the code I use:

import numpy as np

a = np.zeros ((365343020, 9), dtype = np.float32)

for n in range (0, 36534302):
    a[n*10:(n+1)*10,0] = n

where the second row is where I create an array, of 365343020 rows and 9 columns, filled with zeros; while the successive “for” is meant to replace the first column of the array with a vector whose elements are 36534302 sequential integers repeated 10 times each (eg [0,0,…,0,1,1,…,1,2,2,…, 36534301, 36534301,…, 36534301]).

The code seems to respond as desired till around row 168000000 or the array, then it substitute the 10 repetitions of numbers with the last digit odd with a second repetition of the (even) number before.

I have looked for explanations regarding the difference between views and copies. However, even trying to manually define the content of a specific cell of the array (where it is wrongly defined by the loop), it does not change.

Could you please help me in solving this problem? Thanks

Answer 1

Maybe your program is consuming too much memory. Here is some basic math for your code.

Date type: float32
Bits used: 32 bits
Size of array: 3288087180 (365343020*9)
Total memory consumed: 105218789760 bits(13.15234872 GB)

1.Try using float8 bit if value being stored in array is not large.
2.Try to decrease your array size.
3.Both 1 and 2