将 panda dataframe 值组转换为多个列表

Question

I have pandas dataframe, where I listed items, and categorised them:

col_name    |col_group
-------------------------
id          | Metadata
listing_url | Metadata
scrape_id   | Metadata
name        | Text
summary     | Text
space       | Text

To reproduce:

import pandas

df = pandas.DataFrame([
    ['id','metadata'],
    ['listing_url','metadata'],
    ['scrape_id','metadata'],
    ['name','Text'],
    ['summary','Text'],
    ['space','Text']],
    columns=['col_name', 'col_group'])

Can you suggest how I can convert this dataframe to multiple lists based on "col_group":

Metadata = ['id','listing_url','scraping_id]
Text = ['name','summary','space']

This is to allow me to pass these lists of columns to panda and drop columns.

I googled a lot and got stuck: all answers are about converting lists to df, not vice versa. Should I aim to convert into dictionary, or list of lists?

I have over 100 rows, belonging to 10 categories, so would like to avoid manual hard-coding.

Answer 1

Like this:

In [245]: res = df.groupby('col_group', as_index=False)['Col_name'].apply(list)

In [248]: res.tolist()                                                                                                                                                                                      
Out[248]: [['id', 'listing_url', 'scrape_id'], ['name', 'summary', 'space']]

Answer 2

I've try this code:

import pandas

df = pandas.DataFrame([
    [1, 'url_a', 'scrap_a', 'name_a', 'summary_a', 'space_a'],
    [2, 'url_b', 'scrap_b', 'name_b', 'summary_b', 'space_b'],
    [3, 'url_c', 'scrap_c', 'name_c', 'summary_c', 'space_ac']],
    columns=['id', 'listing_url', 'scrape_id', 'name', 'summary', 'space'])
print(df)

for row in df.iterrows():
    print(row[1].to_list())

which give this answer:

[1, 'url_a', 'scrap_a', 'name_a', 'summary_a', 'space_a']
[2, 'url_b', 'scrap_b', 'name_b', 'summary_b', 'space_b']
[3, 'url_c', 'scrap_c', 'name_c', 'summary_c', 'space_ac']

You can use

for row in df[['name', 'summary', 'space']].iterrows():

to only iter over specific columns.

Answer 3

my_vars = df.groupby('col_group').agg(list)['col_name'].to_dict()

Output:

>>> my_vars
{'Text': ['name', 'summary', 'space'], 'metadata': ['id', 'listing_url', 'scrape_id']}

The recommended usage would be just my_vars['Text'] to access the Text , and etc. If you must have this as distinct names you can force it upon your target scope, eg globals :

globals().update(df.groupby('col_group').agg(list)['col_name'].to_dict())

Result:

>>> Text
['name', 'summary', 'space']
>>> metadata
['id', 'listing_url', 'scrape_id']

However I would advise against that as you might unwittingly overwrite some of your other objects, or they might not be in the proper scope you needed (eg locals ).