DataFrame 按两列分组并获取另一列的计数

Question

Novice programmer here seeking help. I have a Dataframe that looks like this:

  Cashtag      Date           Message  
0  $AAPL    2018-01-01   "Blah blah $AAPL"
1  $AAPL    2018-01-05   "Blah blah $AAPL"      
2  $AAPL    2019-01-08   "Blah blah $AAPL"     
3  $AAPL    2019-02-09   "Blah blah $AAPL"
4  $AAPL    2019-02-10   "Blah blah $AAPL"
5  $AAPL    2019-03-01   "Blah blah $AAPL"
6  $FB      2018-01-03   "Blah blah $FB"
7  $FB      2018-02-10   "Blah blah $FB"    
8  $FB      2018-02-11   "Blah blah $FB"   
9  $FB      2019-03-22   "Blah blah $FB" 
10 $AMZN    2018-04-13   "Blah blah $AMZN"
11 $AMZN    2018-04-29   "Blah blah $AMZN"
12 $AMZN    2019-07-23   "Blah blah $AMZN"     
13 $AMZN    2019-07-27   "Blah blah $AMZN"                         

My desired output is a DataFrame that tells me the number of messages for each month of every year in the sample for each company. In this example it would be:

   Cashtag    Date    #Messages       
0  $AAPL    2018-01      02       
1  $AAPL    2019-01      01   
2  $AAPL    2019-02      02     
3  $AAPL    2019-03      01
4  $FB      2018-01      01
5  $FB      2018-02      02        
6  $FB      2019-03      01   
7  $AMZN    2018-04      02  
8  $AMZN    2019-07      02       

I've tried many combinations of.groupby() but have not achieved a solution.

How can I achieve my desired output?

Answer 1

Try:

In case Date is string :

>>> df.groupby([df["Cashtag"], df["Date"].apply(lambda x: x[:7])]).agg({"Message": "count"}).reset_index()

If Date is datetime :

>>> df.groupby([df["Cashtag"], df["Date"].apply(lambda x: "{0}-{1:02}".format(x.year, x.month))]).agg({"Message": "count"}).reset_index()

and output:

  Cashtag     Date  Message
0   $AAPL  2018-01        2
1   $AAPL  2019-01        1
2   $AAPL  2019-02        2
3   $AAPL  2019-03        1
4   $AMZN  2018-04        2
5   $AMZN  2019-07        2
6     $FB  2018-01        1
7     $FB  2018-02        2
8     $FB  2019-03        1

Answer 2

There are two tricky parts. One is handling dates and the other is the groupby itself.

To group by just year and month, you need to extract them from your dates. You can use string indexing, or convert your "Date" column to datetimes and format them with strftime . I will use the second method because I find it more readable and also more useful as a learning point.

The important point about groupby is that you can pass it a list of column labels. Aggregation is then done on every unique combination of values in those columns.

# convert Date to datetimes
df['Date'] = pd.to_datetime(df['Date'])
# extract year and month from datetime objects with `strftime`
df['year-month'] = df['Date'].apply(lambda x: (x.strftime('%Y-%m')))
# groupby columns 'Cashtag' and 'year-month' and aggregate 'Message' using the `count` function
df.groupby(['Cashtag', 'year-month'])['Message'].count()

If you don't want to create a new column, you can do it in a single line:

df.groupby(['Cashtag', df['Date'].apply(lambda x: (x.strftime('%Y-%m')))])['Message'].count()

Answer 3

Solution using resample :

import pandas as pd


data = [
    ('$AAPL', '2018-01-01', "Blah blah $AAPL"),
    ('$AAPL', '2018-01-05', "Blah blah $AAPL"),      
    ('$AAPL', '2019-01-08', "Blah blah $AAPL"),     
    ('$AAPL', '2019-02-09', "Blah blah $AAPL"),
    ('$AAPL', '2019-02-10', "Blah blah $AAPL"),
    ('$AAPL', '2019-03-01', "Blah blah $AAPL"),
    ('$FB',   '2018-01-03', "Blah blah $FB"),
    ('$FB',   '2018-02-10', "Blah blah $FB"),  
]

df = pd.DataFrame.from_records(data=data, columns=['Cashtag', 'Date', 'Message'])


df['Date'] = pd.to_datetime(df['Date'])

df = (df
    .set_index(pd.DatetimeIndex(df['Date']))
    .groupby('Cashtag')
    .resample('M')['Message']
    .count()
    .reset_index()
    .query('Message > 0')
    .reset_index(drop=True)
)
df['Date'] = df['Date'].dt.to_period('M')

Output:

  Cashtag     Date  Message
0   $AAPL  2018-01        2
1   $AAPL  2019-01        1
2   $AAPL  2019-02        2
3   $AAPL  2019-03        1
4     $FB  2018-01        1
5     $FB  2018-02        1

Or even simpler solution:

df['Date'] = pd.to_datetime(df['Date']).dt.to_period('M')
df = df.groupby(['Cashtag', 'Date'])['Message'].count().reset_index()