Concatenating numpy.void with numpy.ndarray

Question

(Warning - I am a newbie)

I imported mat files using scipy.io:

   data = spio.loadmat('data.mat', struct_as_record=True, squeeze_me=True)
   data = data['b']
   >>> <type 'numpy.void'>

Which gave me a file that is of type numpy.void. Each line has 17 entries that are types string, float, array

print(data.shape)
>>> (11000,)

I have another list of strings which I converted to a numpy.array:

filenames = np.array([filenames])
filenames = np.ndarray.flatten(filenames)
print (filenames.shape)
>>> (11000,)

print(data.dtype)
print(filenames.dtype)

>>> [('fieldname1', 'O'), ('fieldname2', 'O'), ('fieldname3', 'O'), ('fieldname4', 'O'), ('fieldname5', 'O'), ('fieldname6', 'O'), ('fieldname7', 'O'), ('fieldname8', 'O'), ('fieldname9', 'O'), ('fieldname10', 'O'), ('fieldname11', 'O'), ('fieldname12', 'O'), ('fieldname13', 'O'), ('fieldname14', 'O'), ('fieldname15', 'O'), ('fieldname16', 'O'), ('fieldname17', 'O')]
>>> |S16

I want to concatenate these along a column:

NEW = np.concatenate((data, filenames), axis=1)

But I am getting this error:

>>> TypeError: invalid type promotion

Any help would be very much appreciated.

Answer 1

recfunctions is a module with tools for fiddling with structured arrays (and their variant, recarray ). It requires separate import. In my experience it is also somewhat buggy.

In [158]: from numpy.lib import recfunctions

Make an array with several object dtype fields:

In [159]: dat = np.empty((3,),dtype=('O,O,O'))
In [160]: dat
Out[160]: 
array([(None, None, None), (None, None, None), (None, None, None)],
      dtype=[('f0', 'O'), ('f1', 'O'), ('f2', 'O')])

After a bit of trial-n-error in calling append_field , this works:

In [161]: names = np.array(['one','two','three'])
In [162]: dat1 = recfunctions.append_fields(dat, 'names', names, usemask=False)
In [163]: dat1
Out[163]: 
array([(None, None, None, 'one'), (None, None, None, 'two'),
       (None, None, None, 'three')],
      dtype=[('f0', 'O'), ('f1', 'O'), ('f2', 'O'), ('names', '<U5')])

But check the contents of the data that's loaded from MATLAB. The .mat may contain structs and cells, which loadmat has to translate into numpy equivalents. To do so it makes extensive use of object dtype arrays.