How should I use Numpy's vstack method?

Question

Firstly, here is the relevant part of the code:

stokes_list = np.zeros(shape=(numrows,1024)) # 'numrows' defined earlier
for i in range(numrows):
    epoch_name = y['filename'][i] # 'y' is an array from earlier
    os.system('pdv -t {0} > temp.txt '.format(epoch_name)) # 'pdv' is a command from another piece of software - here I copy the output into a temporary file
    stokes_line = np.genfromtxt('temp.txt', usecols=3, dtype=[('stokesI','float')], skip_header=1)
    stokes_list = np.vstack((stokes_line,stokes_line))

So, basically, every time the code loops around, stokes_line pulls one of the columns (4th one) from the file temp.txt , and I want it to add a line to stokes_list each time.

For example, if the first stokes_line is

1.1 2.2 3.3  

and the second is

4.4 5.5 6.6  

then stokes_list will be

1.1 2.2 3.3  
4.4 5.5 6.6  

and will keep growing...

It's not working at the moment, because I think that the line:

stokes_list = np.vstack((stokes_line,stokes_line))

is not correct. It's only stacking 2 lists - which makes sense as I only have 2 arguments. I basically would like to know how I keep stacking again and again.

Any help would be very gratefully received!
If it is needed, here is an example of the format of the temp.txt file:

File: t091110_065921.SFTC Src: J1903+0925 Nsub: 1 Nch: 1 Npol: 4 Nbin: 1024 RMS: 0.00118753  
0 0 0 0.00148099 -0.00143755 0.000931365 -0.00296775  
0 0 1 0.000647476 -0.000896698 0.000171287 0.00218597  
0 0 2 0.000704697 -0.00052846 -0.000603842 -0.000868739  
0 0 3 0.000773361 -0.00234724 -0.0004112 0.00358033  
0 0 4 0.00101559 -0.000691062 0.000196023 -0.000163109  
0 0 5 -0.000220367 -0.000944024 0.000181002 -0.00268215  
0 0 6 0.000311783 0.00191545 -0.00143816 -0.00213856  

Answer 1

vstack ing again and again is not good, because it copies the whole arrays.

Create a normal Python list , .append to it and then pass it whole to np.vstack to create a new array once.

stokes_list = []
for i in xrange(numrows):
    ...
    stokes_line = ...
    stokes_list.append(stokes_line)

big_stokes = np.vstack(stokes_list)

Answer 2

You already know the final size of the stokes_list array since you know numrows . So it seems you don't need to grow an array (which is very inefficient). You can simply assign the correct row at each iteration. Simply replace your last line by :

stokes_list[i] = stokes_line

By the way, about your non-working line I think you meant :

stokes_list = np.vstack((stokes_list, stokes_line))

where you're replacing stokes_list by its new value.