Pandas get_dummies in for loop

Question

I would like to convert categorical variables into dummies using pandas.get_dummies in a for loop. However, following code does not convert the dataframes.

data_cleaner = [data_train, data_val]
for df in data_cleaner:
    df = pd.get_dummies(df, columns = categorical_fields)

data_train.head() # Not changed

I know that an iterator in a for loop is just a temporary variable. But the modified code also didn't work.

for i in range(len(data_cleaner)):
    data_cleaner[i] = pd.get_dummies(data_cleaner[i], columns = categorical_fields)

data_train.head() # Still not changed

Anyone can help? Do I have to manually run get_dummies for each dataframe? FYI, Pandas get_dummies doesn't provide an inplace option.

Answer 1

Try following

data_cleaner = [data_train, data_val]
for i,df in enumerate(data_cleaner):
    data_cleaner[i] = pd.get_dummies(df, columns = categorical_fields)

data_train,data_val=data_cleaner

Answer 2

You can run it as a list comprehension

data_cleaner = [pd.get_dummies(df, columns=categorical_fields) for df in data_cleaner]

or

data_train_dum, data_val_dum = [pd.get_dummies(df, columns=categorical_fields) for df in [data_train, data_val]]