How can I properly use pivot on this pandas dataframe?

Question

I have the following df:

Item  Service    Damage    Type          Price
A      Fast       3.5         1          15.48403728
A      Slow       3.5         1          17.41954194
B      Fast        5          1          19.3550466
B      Slow        5          1          21.29055126
C      Fast       5.5         1          23.22605592
and so on

I want to turn this into this format:

Item  Damage  Type   Price_Fast Price_slow

So the first row would be:

Item    Damage     Type    Price_Fast    Price_slow
A        3.5         1        15.4840..     17.41954...

I tried:

df.pivot(index=['Item', 'Damage', 'Type'],columns='Service', values='Price')

but it threw this error:

ValueError: Length of passed values is 2340, index implies 3

Answer 1

To get exactly the dataframe layout you want use

dfData = dfRaw.pivot_table(index=['Item', 'Damage', 'Type'],columns='Service', values='Price')

like @CJR suggested followed by

dfData.reset_index(inplace=True)

to flatten dataframe and

dfData.rename(columns={'Fast': 'Price_fast'}, inplace=True) dfData.rename(columns={'Slow': 'Price_slow'}, inplace=True)

to get your desired column names.

Then use

dfNew.columns = dfNew.columns.values

to get rid of custom index label and your are done (Thanks to @Akaisteph7 for pointing that out that I was not quite done with my previous solution.)

Answer 2

You can do it with the following code:

# You should use pivot_table as it handles multiple column pivoting and duplicates aggregation
df2 = df.pivot_table(index=['Item', 'Damage', 'Type'], columns='Service', values='Price')
# Make the pivot indexes back into columns
df2.reset_index(inplace=True)
# Change the columns' names
df2.rename(columns=lambda x: "Price_"+x if x in ["Fast", "Slow"] else x, inplace=True)
# Remove the unneeded column Index name
df2.columns = df2.columns.values
print(df2)

Output:

  Item  Damage  Type  Price_Fast  Price_Slow
0    A     3.5     1   15.484037   17.419542
1    B     5.0     1   19.355047   21.290551
2    C     5.5     1   23.226056         NaN