numpy array dimension mismatch error

Question

I am quite new to numpy and python in general. I am getting a dimension mismatch error when I try to append values even though I have made sure that both arrays have the same dimension. Also another question I have is why does numpy create a single dimensional array when reading in data from a tab delimited text file.

import numpy as np

names = ["Angle", "RX_Power", "Frequency"]
data = np.array([0,0,0],float)  #experimental

data = np.genfromtxt("rx_power_mode 0.txt", dtype=float, delimiter='\t', names = names, usecols=[0,1,2], skip_header=1)

freq_177 = np.zeros(shape=(data.shape))

print(freq_177.shape)  #outputs(315,)

for i in range(len(data)):

    if data[i][2] == 177:
        #np.concatenate(freq_177,data[i]) has same issue
        np.append(freq_177,data[i],0) 

The output I am getting is

all the input arrays must have same number of dimensions

Answer 1

Annotated code:

import numpy as np

names = ["Angle", "RX_Power", "Frequency"]

You don't need to 'initialize' an array - unless you are going to assign values to individual elements.

data = np.array([0,0,0],float)  #experimental

This data assignment completely overwrites the previous one.

data = np.genfromtxt("rx_power_mode 0.txt", dtype=float, delimiter='\t', names = names, usecols=[0,1,2], skip_header=1)

Look at data at this point. What is data.shape ? What is data.dtype ? Print it, or at least some elements. With names I'm guessing that this is a 1d array, with a 3 field dtype . It's not a 2d array, though, with all floats it could transformed/view as such.

Why are you making a 1d array of zeros?

freq_177 = np.zeros(shape=(data.shape))
print(freq_177.shape)  #outputs(315,)

With a structured array like data , the preferred way to index a given element is by field name and row number, eg. data['frequency'][i]`. Play with that.

np.append is not the same as the list append. It returns a value; it does not change freq_177 in place. Same for concatenate . I recommend staying away from np.append . It's too easy to use it in the wrong way and place.

for i in range(len(data)):
    if data[i][2] == 177:
        #np.concatenate(freq_177,data[i]) has same issue
        np.append(freq_177,data[i],0) 

It looks like you want to collect in freq_177 all the terms of the data array for which the 'frequency' field is 177.

I = data['frequency'].astype(int)==177
freq_177 = data[I]

I have used astype(int) because the == test with floats is uncertain. It is best used with integers.

I is a boolean mask, true where the values match; data[I] then is the corresponding elements of data . The dtype will match that of data , that is, it will have 3 fields. You can't append or concatenate it to an array of float zeros (your original freq_177 ).

If you must iterate and collect values, I suggest using list append, eg

alist = []
for row in data:
   if int(row['frequency'])==177:
       alist.append(row)
freq177 = np.array(alist)

---

I don't think np.append is discussed much except in its own doc page and text. It comes up periodically in SO questions.

http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.append.html

Returns: append : ndarray

A copy of arr with values appended to axis. Note that append does not occur in-place: a new array is allocated and filled.

See also help(np.append) in an interpreter shell.

For genfromtxt - it too has docs, and lots of SO discussion. But to understand what it returned in this case, you need to also read about structured arrays and compound dtype . (add links?)

Try loading the data with:

data = np.genfromtxt("rx_power_mode 0.txt", dtype=float, delimiter='\t', usecols=[0,1,2], skip_header=1)

Since you are skipping the header line, and just using columns with floats, data should be a 2d array with 3 columns, (N, 3) . In that case you could access the 'frequency' values with data[:,2]

I = int(data[:,2])==177
freq_177 = data[I,:]

freq_177 is now be a 3 column array - with a subset of the data rows.