numpy array inhomogeneous shape 2d

Question

training = []
empty_list = [0] *len(classes)

for doc in docs:
    bag = []
    word_pat = doc[0]
    word_pat = [lem.lemmatize((w.lower())) for w in word_pat] 
    for w in words:
        if w in word_pat:
            bag.append(1)
        else:
            bag.append(0)
    output_row = list(empty_list)
    output_row[classes.index(doc[1])] = 1
    training.append([bag, output_row])

random.shuffle(training)
**training = np.array(training)**

after executing, the following error pops up:

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (38, 2) + inhomogeneous part.

any ideas?

I was mostly following a tutorial i saw online, but even if i follow step by step, it doesnt work, it may be outdated. I have only recently looked into ML etc and done mostly java OOP before, so i dont really have the knowledge to fix this error cause i dont completly understand numpy.

Answer 1

What numpy version are you using?

Normally I'd ask for the full error message with traceback. But if the ** is supposed to indicate the problem line, that may be enough (if non conventional).

In an older numpy version I could recreate a similar problem with:

In [171]: alist = [np.ones((3,4,2)), np.zeros((3,4,3))]
In [172]: np.array(alist)
C:\Users\paul\AppData\Local\Temp\ipykernel_6836\2629805649.py:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  np.array(alist)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[172], line 1
----> 1 np.array(alist)

ValueError: could not broadcast input array from shape (3,4,2) into shape (3,4)

The error is self explanatory. You are trying to combine into one array several arrays that having incompatible shapes.

Have you check the shapes of the elements of the training list?

Answer 2

The training array is not in the correct format. You will need to pass an extra parameter to the method to make it work.

Replace your asterisked code with this:

training = np.array(training, dtype=object)