python numpy split array into unequal subarrays

Question

I am trying to split an array into n parts. Sometimes these parts are of the same size, sometimes they are of a different size.

I am trying to use:

split = np.split(list, size)

This works fine when size divides equally into the list, but fails otherwise. Is there a way to do this which will 'pad' the final array with the extra 'few' elements?

Answer 1

Are you looking for np.array_split? Here is the docstring:

Split an array into multiple sub-arrays.

Please refer to the ``split`` documentation.  The only difference
between these functions is that ``array_split`` allows
`indices_or_sections` to be an integer that does *not* equally 
divide the axis.

See Also
--------
split : Split array into multiple sub-arrays of equal size.

Examples
--------
>>> x = np.arange(8.0)
>>> np.array_split(x, 3)
    [array([ 0.,  1.,  2.]), array([ 3.,  4.,  5.]), array([ 6.,  7.])]

http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.array_split.html

Answer 2

def split_padded(a,n):
    padding = (-len(a))%n
    return np.split(np.concatenate((a,np.zeros(padding))),n)

Answer 3

short method: use : numpy.array_split instead of numpy.split

but the best way to go about this is to split the array using partial splits

def cs(chunksize, fs):
    sa = []
    num =chunksize
    while(num< fs):
        sa.append(num)
        num += chunksize
    sa.append(num+fs%chunksize)
    return sa

pass the function as a parameter in the split

for chunk in np.split(df, cs(chunksize,fs)):
    chunk.to_excel('{}/{}_{:02d}.xlsx'.format(output_folder,split_name, i), index=False)
    i +=1

Answer 4

You can split arrays into unequal chunks by passing indices as a list Example

**x = np.arange(10)**
x
(0,1,2,3,4,5,6,7,8,9)
np.array_split(x,[4])
[array([0,1,2,3],dtype = int64),
       array([4,5,6,7,8,9],dtype = int64)**