How can I implement a dictionary whose value is mutable length array in numpy

Question

I want to use numpy to implement the following data structure. Now I use python dictionary to do the work, but it's hard to do the vector operations, I have to add the vector many times, so I want to use numpy to simplify the work. The length of hosts will vary during program execution. Is it possible for me to do this job with numpy structured arrays, notice that the length of list is mutable? I'm not familiar with it, just want to know whether it's possible, so that it won't be a waste of time.

{
  "0" :{
      "coordinates": [100, 100],
      "neighbours": [1, 40],
      "hosts":[],
      "v-capacity":20,
      "v-immature":0,
      "v-state":[20, 0, 0, 0]
  },
  "1" :{
      "coordinates": [200, 100],
      "neighbours": [0, 2, 41],
      "hosts":[],
      "v-capacity":20,
      "v-immature":0,
      "v-state":[20, 0, 0, 0]
  },

Answer 1

What you show is a dictionary whose values are also dictionaries. Some values of the nested dictionaries are scalars, others are lists. neighbors list varies in length.

I can picture creating a structured array with fields corresponding to the inner dictionary keys.

The coordinates and v-state fields could even have inner dimensions of (2,) and (4,).

But for variable length neighbors or hosts the best we can do it define those fields as having object dtype, which will store the respective lists elsewhere in memory. Math on that kind of array is limited.

But before you get too deep into structured arrays, explore creating a set of arrays to store this data, one row per item in the out dictionary.

coordinates = np.array([[100,100],[200,100]])
neighbors = np.array([[1, 40],[0, 2, 41]])

Make sure you understand what those expressions produce.

In [537]: coordinates
Out[537]: 
array([[100, 100],
       [200, 100]])
In [538]: neighbors
Out[538]: array([[1, 40], [0, 2, 41]], dtype=object)

Here's an example of a structured array that can hold these arrays:

In [539]: dt=np.dtype([('coordinates',int,(2,)),('neighbors',object)])
In [540]: arr = np.zeros((2,), dtype=dt)
In [541]: arr
Out[541]: 
array([([0, 0], 0), ([0, 0], 0)], 
      dtype=[('coordinates', '<i4', (2,)), ('neighbors', 'O')])
In [543]: arr['coordinates']=coordinates
In [544]: arr['neighbors']=neighbors
In [545]: arr
Out[545]: 
array([([100, 100], [1, 40]), ([200, 100], [0, 2, 41])], 
      dtype=[('coordinates', '<i4', (2,)), ('neighbors', 'O')])
In [546]: arr['neighbors']
Out[546]: array([[1, 40], [0, 2, 41]], dtype=object)

Notice that is basically a packaging convenience. It stores the arrays in one place, but you still have perform your math/vector operations on the individual fields.

In [547]: coordinates.sum(axis=1)
Out[547]: array([200, 300])     # sum across columns of a 2d array
In [548]: neighbors.sum()
Out[548]: [1, 40, 0, 2, 41]    # sum (concatenate) of lists