将 groupedby pandas 数据框（多列但不是所有列）从长转换为宽

Question

The problem:

I have a dataset with yearly data of different companies. The data is stored in a long format, each year is a row therefore company ids are duplicated. The data looks like this (however in the original dataframe I have lot more columns).

I would need to transform the long type format to wide type format, so each company will be shown in one row (no duplication)

This is the result I would like to look like:

As you can see I would need:

• some columns (like "year") are not needed any more

• some columns (like "sales", "sales_change_in_2_years", "sales_change_over_year") should be transformed from wide format to long format and keeping their names (and adding a number to them)

• some columns (like "ind1" and "ind2") should remain as they are (no transformation from wide to long)

So far I was able to workout a solution which works only on one columns, so it is really not a solution for me.

This is my code:

test.groupby("comp_id")['sales_change_1'].apply(list).apply(pd.Series).rename(columns=lambda x: 'sales_{}'.format(x+1))

Is there a better solution to my problem?

Answer 1

After you drop the years:

del test['Year']

You can manage to group the lines together by adding an extra column with the row "index" for each row belonging to the same company.

test['idx'] = test.groupby('Comp_id').cumcount() + 1

Then set it as part as the DataFrame index and use unstack() to turn it into columns.

test = test.set_index(['Comp_id', 'idx']).unstack()

At this point, your columns will be a MultiIndex with the created 'idx' as a second level, so you could already use the DataFrame as it stands referring to columns as ('Sales', 1) , ('Sales', 2) , etc.

If you want to flatten your columns, using underscore as the separator, you can do so with:

test.columns = ['{}_{}'.format(col, idx) for (col, idx) in test.columns]