How to determine the length of lists of list in a pandas dataframe column?

Question

I have a data frame with a column containing a list of lists like this.

index	('source', 'target')	value of shortest path	list of paths
0	('a', 'b')	3	[['a', 'c', 'b'], ['a', 'c', 'd', 'b'], ['a', 'e', 'f', 'g', 'b']]
1	('g', 'z')	1	[['g', 'z'], ['g', 'l', 'z']]

I want to create 2 columns in the first one I want to have a list of lengths of paths for each list in this list except if the length of the path is equal to the length of the shortest path (length of paths is equal of the size of each list minus 1). In the second column I want to sum reversed values for each row in the 'length of paths' column.

lenght of paths	total value
[3, 4]	0.583
[2]	0.5

The code I tried to use was:

path_lenght = []
for i, row in df.iterrows():
    for k in df['list of paths']:
        if (len(k)-1) > row['shorthest_path_value']:
            path_lenght.append(len(k)-1)
       
    df['lenght of paths'] = path_lenght

However, this returned the following error:

ValueError: Length of lenght of paths (243007) does not match length of index (252454)

How can I solve this problem?

Answer 1

You can create 2 functions, one for each column and apply them with apply() method. See below:

def length_of_paths(l):
    k=[len(i) for i in l]
    minlen=min(k)
    return [i-1 for i in k if i!=minlen]

def total_value(l):
    return sum([1/i for i in l])

df['length of paths']=df['list of paths'].apply(lambda x: length_of_paths(x))

df['total value']=df['length of paths'].apply(lambda x: total_value(x))

Output:

>>> print(df)

                                list of paths length of paths  total value
0  [[a, c, b], [a, c, d, b], [a, e, f, g, b]]          [3, 4]     0.583333
1                         [[g, z], [g, l, z]]             [2]     0.500000