How do I calculate the mean of elements in a list across columns?

Question

Say I have an array of lists

B = [[1,2,3],[1,2,3,4],[1,2]]

and I want to calculate the mean of the elements in the lists across the coloumns. How do I do this?

ie

How do I get a final array of mean values equal to the longest list:

[(1+1+1)/3,(2+2+2)/3,(3+3)/2,4/1] = [1,2,3,4]

I tried:

final_array = np.array([mean(a) for a in zip(*(B))])

But this gives me an array only as long as my shortest list. Is this were masks come in handy? My apologies if an array of lists makes you cringe, I am still getting used to Python.

Answer 1

You can it by using DataFrame of pandas.

from pandas import DataFrame

B = [[1,2,3],[1,2,3,4],[1,2]]
df = DataFrame(B)
df.mean(axis=0)
""""
df
   0  1   2   3
0  1  2   3 NaN
1  1  2   3   4
2  1  2 NaN NaN

df.mean(axis=0)
0    1
1    2
2    3
3    4
"""

Answer 2

You'll need to pad your list with some sentinel value (I used NaN) and then create a masked array with that sentinel. Once you have the masked array, you can calculate the mean without any problem.

>>> import numpy as np
>>> B = [[1,2,3],[1,2,3,4],[1,2]]
>>> 
>>> maxlen = max(len(x) for x in B)
>>> C = np.array([l+[np.nan]*(maxlen-len(l)) for l in B])
>>> C
array([[  1.,   2.,   3.,  nan],
       [  1.,   2.,   3.,   4.],
       [  1.,   2.,  nan,  nan]])
>>> dat = np.ma.fix_invalid(C)
>>> np.mean(dat,axis=0)
masked_array(data = [1.0 2.0 3.0 4.0],
             mask = [False False False False],
       fill_value = 1e+20)

Answer 3

Using itertools.izip_longest and itertools.takewhile :

>>> from itertools import takewhile, izip_longest
def means(lis):
    fill = object()
    for item in izip_longest(*lis,fillvalue = fill):
        vals = list(takewhile( lambda x : x!=fill , item))
        yield sum(vals)/float(len(vals))
...         
>>> lis = [[1,2,3],[1,2,3,4],[1,2]]
>>> lis.sort( key = len, reverse = True) #reverse sort the list based on length of items
>>> list(means(lis))
[1.0, 2.0, 3.0, 4.0]

Answer 4

One more way, using cmp and izip_longest

from itertools import izip_longest
[float(sum(col)) / sum(cmp(x,0) for x in col) for col in izip_longest(*B, fillvalue=0)]

This assumes your values are positive.

Answer 5

B = [[1,2,3],[1,2,3,4],[1,2]]
data = {}
max_len = 0

for alist in B:
    length = len(alist)
    max_len = length if (length > max_len) else max_len

    for i in range(length):
        data.setdefault(i, []).append(alist[i])


results = []

for i in range(max_len):
    vals = data[i]
    results.append(sum(vals) / len(vals) )

print results

--output:--
[1, 2, 3, 4]

Answer 6

You can do it without any external libs:

B = [[1,2,3],[1,2,3,4],[1,2]]
#compute max length of sub list
maxLen = max([len(x) for x in B])
#new list with number of empty lists equals to number of columns
transList = [[] for i in range(maxLen)]
#transforming list to new structure
for row in B:
    for col in row:
        transList[col-1].append(col)
#transList = [[1, 1, 1], [2, 2, 2], [3, 3], [4]] from now one its simple to get mean of the elements ;)
meanB = [float(sum(i))/len(i) for i in transList]