How to add several vectors to numpy structered array and call matrix later from fieldname?

Question

Hey guys Ii need help..

I want to use tensorflows data import, where data is loaded by calling the features/labels vectors from a structured numpy array.

https://www.tensorflow.org/programmers_guide/datasets#consuming_numpy_arrays

I want to create such an structured array by adding consecutively the 2 vectors (feature_vec and label_vec) to an numpy structured array.

import numpy as np

# example vectors
feature_vec= np.arange(10)
label_vec = np.arange(10)

# structured array which should get the vectors
struc_array = np.array([feature_vec,label_vec],dtype=([('features',np.float32), ('labels',np.float32)]))

# How can I add now new vectors to struc_array?

struc_array.append(---)

I want later when this array is loaded from file call either the feature vectors (which is a matrix now) by using the fieldname:

with np.load("/var/data/training_data.npy") as data:
features = data["features"] # matrix containing feature vectors as rows
labels = data["labels"] #matrix containing labels vectors as rows

Everything I tried to code was complete crap.. never got a correct output..

Thanks for your help!

Answer 1

Don't create a NumPy array and then append to it. That doesn't really make sense, as NumPy arrays have a fixed size and require a full copy to append a single row or column. Instead, create a list, append to it, then construct the array at the end:

vecs = [feature_vec,label_vec]
dtype = [('features',np.float32), ('labels',np.float32)]

# append as many times as you want:
vecs.append(other_vec)
dtype.append(('other', np.float32))

struc_array = np.array(vecs, dtype=dtype)

Of course, you probably need ot

Answer 2

Unfortunately, this doesn't solve the problem.

i want to get just the labels or the features from structured array by using:

labels = struc_array['labels']
features = struc_array['features']

But when i use the structured array like you did, labels and also features contains all given appended vectors:

import numpy as np

feature_vec= np.arange(10)
label_vec = np.arange(0,5,0.5)

vecs = [feature_vec,label_vec]
dtype = [('features',np.float32), ('labels',np.float32)]

other_vec = np.arange(6,11,0.5)

vecs.append(other_vec)
dtype.append(('other', np.float32))

struc_array = np.array(vecs, dtype=dtype)


# This contains all vectors.. not just the labels vector
labels = struc_array['labels']

# This also contains all vectors.. not just the feature vector
features = struc_array['features']