loop over series of dictionaries to convert to DataFrames

Question

I have that list of dictionaries in several rows that I need to loop over to create new DataFrame. I have tried the following loop:

 for key, value in s.items():

    print(key + " : " + str(value))

but getting the following error: unsupported operand type(s) for +: 'int' and 'str'

The type of my s: pandas.core.series.Series

Convert string-encoded list into new dataframe

s:

0        [{'cast_id': 14, 'character': 'Woody (voice)',...
1        [{'cast_id': 14, 'character': 'Woody (voice)',...
2        [{'cast_id': 14, 'character': 'Woody (voice)',...
45474    [{'cast_id': 14, 'character': 'Woody (voice)',...
45475    [{'cast_id': 14, 'character': 'Woody (voice)',...
Name: cast, Length: 45476, dtype: object

Answer 1

The error suggests that you are looping over strings and not dictionaries. Could you be a bit more specific as to how your data looks?

EDIT: probably not the best way to do it but:

for d in s:
    for key, value in d.items():
        print(key + ": " + str(value))

Answer 2

After reading the related post and your comment, I assume that you have a pandas Series (of a column from a DataFrame which is the same) containing string representations of lists of dicts.

Conceptualy what should be done will use the following conversions:

• string -> list of dicts: I would use ast.literal_eval
• list(s) of dicts -> DataFrame(s) -> a DataFrame can directly be created from a list of dicts.

So you could build a list of DataFrames, one per row of your Series that way:

dfs = [pd.DataFrame(ast.literal_eval(d)) for d in s]

you could even concatenate all of them in one single dataframe:

df = pd.concat(dfs, ignore_index=True)

or directly build the full dataframe with:

df = pd.DataFrame([d for elt in s for d in ast.literal_eval(elt)])