Pandas: Condition based on list in a cell

Question

Dataframe looks like this (blank cells are '', field, extra_dimensions are columns)

field | extra_dimensions
------------------------
a     | 
b     | [abc, def]
c     | [ghi]

I have a list of required dimensions and extra dimensions:

required_dimensions = [123, 456]
extra_dimensions = [abc, def, ghi]

Desired output:

field | 123 | 456 | abc | def | ghi
-----------------------------------
a     | 1   | 1   | 0   | 0   | 0
b     | 1   | 1   | 1   | 1   | 0
c     | 1   | 1   | 0   | 0   | 1

Attempt:

columns = ['field', 'extra_dimensions'] + required_dimensions + extra_dimensions
df = df.reindex(columns=columns)
for i in required_dimensions:
    df[i].fillna('1', inplace=True)
for i in extra_dimensions:
    df[i][df['extra_dimensions'].str.contains(i)] = '1'

But I get:

ValueError: cannot index with vector containing NA / NaN values

Would love any input on my attempt or any ideas on a better approach. Thanks in advance!

Answer 1

By using get_dummies again .....

required_dimensions = ['123', '456']
df=pd.DataFrame({'field':list('abc'),'extra_dimensions':[[],['abc','def'],['ghi']]})
df=pd.get_dummies(df.set_index('field')['extra_dimensions'].apply(pd.Series).stack()).sum(level=0).reindex(df.field).fillna(0)
d = dict.fromkeys(required_dimensions, 1)
df.assign(**d)

Out[283]: 
       abc  def  ghi  123  456
field                         
a      0.0  0.0  0.0    1    1
b      1.0  1.0  0.0    1    1
c      0.0  0.0  1.0    1    1