Python Dynamic Array allocation, Matlab style

Question

I'm trying to move a few Matlab libraries that I've built to the python environment. So far, the biggest issue I faced is the dynamic allocation of arrays based on index specification. For example, using Matlab, typing the following:

x = [1 2];
x(5) = 3;

would result in:

x = [ 1     2     0     0     3]

In other words, I didn't know before hand the size of (x), nor its content. The array must be defined on the fly, based on the indices that I'm providing.

In python, trying the following:

from numpy import *
x = array([1,2])
x[4] = 3

Would result in the following error: IndexError: index out of bounds. On workaround is incrementing the array in a loop and then assigned the desired value as :

from numpy import *
x = array([1,2])

idx = 4
for i in range(size(x),idx+1):
    x = append(x,0)

x[idx] = 3
print x

It works, but it's not very convenient and it might become very cumbersome for n-dimensional arrays.I though about subclassing ndarray to achieve my goal, but I'm not sure if it would work. Does anybody knows of a better approach?

---

Thanks for the quick reply. I didn't know about the setitem method (I'm fairly new to Python). I simply overwritten the ndarray class as follows:

import numpy as np

class marray(np.ndarray):

    def __setitem__(self, key, value):

        # Array properties
        nDim = np.ndim(self)
        dims = list(np.shape(self))

        # Requested Index
        if type(key)==int: key=key,
        nDim_rq = len(key)
        dims_rq = list(key)

        for i in range(nDim_rq): dims_rq[i]+=1        

        # Provided indices match current array number of dimensions
        if nDim_rq==nDim:

            # Define new dimensions
            newdims = []
            for iDim in range(nDim):
                v = max([dims[iDim],dims_rq[iDim]])
                newdims.append(v)

            # Resize if necessary
            if newdims != dims:
              self.resize(newdims,refcheck=False)

        return super(marray, self).__setitem__(key, value)

And it works like a charm! However, I need to modify the above code such that the setitem allow changing the number of dimensions following this request:

a = marray([0,0])
a[3,1,0] = 0

Unfortunately, when I try to use numpy functions such as

self = np.expand_dims(self,2)

the returned type is numpy.ndarray instead of main .marray. Any idea on how I could enforce that numpy functions output marray if a marray is provided as an input? I think it should be doable using array_wrap , but I could never find exactly how. Any help would be appreciated.

Answer 1

Took the liberty of updating my old answer from Dynamic list that automatically expands . Think this should do most of what you need/want

class matlab_list(list):
    def __init__(self):
        def zero():
            while 1:
                yield 0
        self._num_gen = zero()

    def __setitem__(self,index,value):
        if isinstance(index, int):
            self.expandfor(index)
            return super(dynamic_list,self).__setitem__(index,value)

        elif isinstance(index, slice):
            if index.stop<index.start:
                return super(dynamic_list,self).__setitem__(index,value)
            else:
                self.expandfor(index.stop if abs(index.stop)>abs(index.start) else index.start)
            return super(dynamic_list,self).__setitem__(index,value)

    def expandfor(self,index):
            rng = []
            if abs(index)>len(self)-1:
                if index<0:
                    rng = xrange(abs(index)-len(self))
                    for i in rng:
                        self.insert(0,self_num_gen.next())
                else:
                    rng = xrange(abs(index)-len(self)+1)
                    for i in rng:
                        self.append(self._num_gen.next())

# Usage
spec_list = matlab_list()
spec_list[5] = 14

Answer 2

This isn't quite what you want, but...

x = np.array([1, 2])

try:
    x[index] = value
except IndexError:
    oldsize = len(x)   # will be trickier for multidimensional arrays; you'll need to use x.shape or something and take advantage of numpy's advanced slicing ability
    x = np.resize(x, index+1) # Python uses C-style 0-based indices
    x[oldsize:index] = 0 # You could also do x[oldsize:] = 0, but that would mean you'd be assigning to the final position twice.
    x[index] = value

>>> x = np.array([1, 2])
>>> x = np.resize(x, 5)
>>> x[2:5] = 0
>>> x[4] = 3
>>> x
array([1, 2, 0, 0, 3])

Due to how numpy stores the data linearly under the hood (though whether it stores as row-major or column-major can be specified when creating arrays), multidimensional arrays are pretty tricky here.

>>> x = np.array([[1, 2, 3], [4, 5, 6]])
>>> np.resize(x, (6, 4))
array([[1, 2, 3, 4],
       [5, 6, 1, 2],
       [3, 4, 5, 6],
       [1, 2, 3, 4],
       [5, 6, 1, 2],
       [3, 4, 5, 6]])

You'd need to do this or something similar:

>>> y = np.zeros((6, 4))
>>> y[:x.shape[0], :x.shape[1]] = x
>>> y
array([[ 1.,  2.,  3.,  0.],
       [ 4.,  5.,  6.,  0.],
       [ 0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.]])

Answer 3

A python dict will work well as a sparse array. The main issue is the syntax for initializing the sparse array will not be as pretty:

listarray = [100,200,300]
dictarray = {0:100, 1:200, 2:300}

but after that the syntax for inserting or retrieving elements is the same

dictarray[5] = 2345