List of lists to rows in a pandas dataframe

Question

After using my script my algorithms return the exptected outcome in a list of lists like this: pred=[[b,c,d],[b,a,u],...[b,i,o]]

I already have a dataframe that needs those values added in a new matching column. The list is exactly x long like the other columns in the frame and I just need to create a new column with all the values of the lists.

However when I try to put the list into the column I get the error:

ValueError: Length of values does not match length of index

Looking at the data, it puts the entire list into one row instead of each entry in a new row.

EDIT:

All values in the list should be put in the column namend pred

sent  token   pred
 0     a        b
 0     b        c
 0     b        d
 1     a        b
 1     b        a
 1     c        u

Solution:

x = []
for _ in pred:
  if _ is not None:
    x += _

df_new = pd.DataFrame(df)
df_new["pred"] = list(itertools.chain.from_iterable(x))

Answer 1

import pandas as pd

# combine input lists
x = []
for _ in [['b','c','d'],['b','a','u'], ['b','i','o']]:
    x += _

# output into a single column
a = pd.Series(x)

# mock original dataframe
b = pd.DataFrame({'sent': [0, 0, 0, 1, 1, 1], 
                  'token': ['a', 'b', 'b', 'a', 'b', 'c']})

# add column to existing dataframe
# this will avoid the mis matched length error by ignoring anything longer 
# than your original data frame
b['pred'] = a

   sent token pred
0     0     a    b
1     0     b    c
2     0     b    d
3     1     a    b
4     1     b    a
5     1     c    u

Answer 2

You can use itertools.chain , which allows you to flatten a list of lists, which you can then slice according to the length of your dataframe.

Data from @ak_slick.

import pandas as pd
from itertools import chain

df = pd.DataFrame({'sent': [0, 0, 0, 1, 1, 1], 
                   'token': ['a', 'b', 'b', 'a', 'b', 'c']})

lst = [['b','c',None],['b',None,'u'], ['b','i','o']]

df['pred'] = list(filter(None, chain.from_iterable(lst)))[:len(df.index)]

print(df)

   sent token pred
0     0     a    b
1     0     b    c
2     0     b    d
3     1     a    b
4     1     b    a
5     1     c    u