如何使用Pandas将共享数据的多行CSV组合成一行？

Question

I have downloaded the ASCAP database , giving me a CSV that is too large for Excel to handle. I'm able to chunk the CSV to open parts of it, the problem is that the data isn't super helpful in its default format. Each song title has 3+ rows associated with it:

The first row include the % share that ASCAP has in that song. The rows after that include a character code (ROLE_TYPE) that indicates if that row contains the writer or performer of that song. The first column of each row contains a song title.

This structure makes the data confusing because on the rows that list the % share there are blank cells in the NAME column because that row does not have a Writer/Performer associated with it.

What I would like to do is transform this data from having 3+ rows per song to having 1 row per song with all relevant data.

So instead of:

TITLE, ROLE_TYPE, NAME, SHARES, NOTE

I would like to change the data to:

TITLE, WRITER, PERFORMER, SHARES, NOTE

Here is a sample of the data:

TITLE,ROLE_TYPE,NAME,SHARES,NOTE
SCORE MORE,ASCAP,Total Current ASCAP Share,100,
SCORE MORE,W,SMITH ANTONIO RENARD,,
SCORE MORE,P,SMITH SHOW PUBLISHING,,
PEOPLE KNO,ASCAP,Total Current ASCAP Share,100,
PEOPLE KNO,W,SMITH ANTONIO RENARD,,
PEOPLE KNO,P,SMITH SHOW PUBLISHING,,
FEEDBACK,ASCAP,Total Current ASCAP Share,100,
FEEDBACK,W,SMITH ANTONIO RENARD,,

I would like the data to look like: TITLE, WRITER, PERFORMER, SHARES, NOTE SCORE MORE, SMITH ANTONIO RENARD, SMITH SHOW PUBLISHING, 100, PEOPLE KNO, SMITH ANTONIO RENARD, SMITH SHOW PUBLISHING, 100, FEEDBACK, SMITH ANONIO RENARD, SMITH SHOW PUBLISHING, 100,

I'm using python/pandas to try and work with the data. I am able to use groupby('TITLE') to group rows with matching titles.

import pandas as pd

data = pd.read_csv("COMMA_ASCAP_TEXT.txt", low_memory=False)

title_grouped = data.groupby('TITLE')

for TITLE,group in title_grouped:
  print(TITLE)
  print(group)

I was able to groupby('TITLE') of each song, and the output I get seems close to what I want:

SCORE MORE
   TITLE          ROLE_TYPE  NAME                        SHARES    NOTE
0  SCORE MORE     ASCAP      Total Current ASCAP Share   100.0     NaN
1  SCORE MORE         W      SMITH ANTONIO RENARD        NaN       NaN
2  SCORE MORE         P      SMITH SHOW PUBLISHING       NaN       NaN 

What do I need to do to take this group and produce a single row in a CSV file with all the data related to each song?

Answer 1

I would recommend:

• Decompose the data by the ROLE_TYPE
• Prepare the data for merge (rename columns and drop unnecessary columns)
• Merge everything back into one DataFrame

Merge will be automatically performed over the column which has the same name in the DataFrames being merged (TITLE in this case).

Seems to work nicely :)

data = pd.read_csv("data2.csv", sep=",")

# Create 3 individual DataFrames for different roles
data_ascap = data[data["ROLE_TYPE"] == "ASCAP"].copy()
data_writer = data[data["ROLE_TYPE"] == "W"].copy()
data_performer = data[data["ROLE_TYPE"] == "P"].copy()

# Remove unnecessary columns for ASCAP role
data_ascap.drop(["ROLE_TYPE", "NAME"], axis=1, inplace=True)

# Rename columns and remove unnecesary columns for WRITER role
data_writer.rename(index=str, columns={"NAME": "WRITER"}, inplace=True)
data_writer.drop(["ROLE_TYPE", "SHARES", "NOTE"], axis=1, inplace=True)

# Rename columns and remove unnecesary columns for PERFORMER role
data_performer.rename(index=str, columns={"NAME": "PERFORMER"}, inplace=True)
data_performer.drop(["ROLE_TYPE", "SHARES", "NOTE"], axis=1, inplace=True)

# Merge all together
result = data_ascap.merge(data_writer, how="left")
result = result.merge(data_performer, how="left")

# Print result
print(result)