Python Pandas - Group by, then plot by category

Question

Very easy pandas question, I'm a beginner.

I have a dataframe 'df' with (for example):

import pandas as pd
df = pd.DataFrame({'time': ['2019-04-23 10:21:00', '2019-04-23 11:14:00', '2019-04-24 11:30'], 
                   'category': ['A', 'B', 'A'],
                   'text': ['njrnfrjn','fmrjfmrfmr','mjrnfjrnmi']})

I just want to:

• Group by category and dates (daily)
• Count the number of text message by category and day
• Plot all timeseries across days (one timeseries for each category in the same plot)

Thanks

Answer 1

You can try the following:

df.groupby([df.time.dt.floor('d'), "category"]).size().unstack().plot()

Explanations :

• First step is to grouby as you mentioned. To do this, we use groupby

• In the groupby , because we need to group the times by days, one solution is to use dt.floor on the time column. We pass the argument "d" for days .

  • Also, to be sure the floor is reachable, the time column must be a time series . If it's not, use pd.to_datetime to convert it with pd.to_datetime(df.time) .

• Now we have the group, the size can be easily computed applying the size method.

• The next step is to convert the category column (at this step as index) into columns. Because we groupby by two keys, we can use unstack .

• Finally, call the plot one the dataframe. Because the dataframe is well structured, it works without any arguments (one line is drawn for each column and the index column ( time ) is used as x-axis .

---

Full code + illustration :

# import modules 
import pandas as pd
import matplotlib.pyplot as plt
# (here random is just for creating dummy data)
from random import randint, choice

# Create dummy data
size = 1000
df = pd.DataFrame({
    'time': pd.to_datetime(["2020/01/{} {}:{}".format(randint(1, 31), randint(0,23), randint(0,59)) for _ in range(size)]),
    'text': ['blablabla...' for _ in range(size)],
    'category': [choice(["A", "B", "C"]) for _ in range(size)]
})
print(df)
#                    time          text category
# 0   2020-01-30 23:15:00  blablabla...        C
# 1   2020-01-16 07:06:00  blablabla...        A
# 2   2020-01-03 18:47:00  blablabla...        A
# 3   2020-01-21 15:45:00  blablabla...        A
# 4   2020-01-10 04:11:00  blablabla...        C
# ..                  ...           ...      ...
# 995 2020-01-12 03:03:00  blablabla...        C
# 996 2020-01-08 10:35:00  blablabla...        B
# 997 2020-01-24 20:51:00  blablabla...        C
# 998 2020-01-05 07:39:00  blablabla...        A
# 999 2020-01-26 16:54:00  blablabla...        A

# See size result
print(df.groupby([df.time.dt.floor('d'), "category"]).size())
# time        category
# 2020-01-01  A            6
#             B           18
#             C            7
# 2020-01-02  A           10
#             B            8
#                         ..
# 2020-01-30  B           16
#             C           11
# 2020-01-31  A           14
#             B           17
#             C           11

# See unstack result
print(df.groupby([df.time.dt.floor('d'), "category"]).size().unstack())
# category     A   B   C
# time
# 2020-01-01   6  18   7
# 2020-01-02  10   8  13
# 2020-01-03  11  11  16
# 2020-01-04   9   5  10
# 2020-01-05  13   9  13
# 2020-01-06  11  11  12
# 2020-01-07  13   7   9
# 2020-01-08   5  16  13
# 2020-01-09  15   6  14
# 2020-01-10  10  11   9
# 2020-01-11   7  16  13
# 2020-01-12  12  13  13
# 2020-01-13  12   5   7
# 2020-01-14  11  10  11
# 2020-01-15  13  14  11
# 2020-01-16   9   8  13
# 2020-01-17   8   9   6
# 2020-01-18  12   5  11
# 2020-01-19   7   8  13
# 2020-01-20  12   9   9
# 2020-01-21   9  13  13
# 2020-01-22  14  11  19
# 2020-01-23  14   6  12
# 2020-01-24   7   8   6
# 2020-01-25  10  12  10
# 2020-01-26   8  12   7
# 2020-01-27  18  11   7
# 2020-01-28  15  10   9
# 2020-01-29  12   7  11
# 2020-01-30  12  16  11
# 2020-01-31  14  17  11

# Perform plot
df.groupby([df.time.dt.floor('d'), "category"]).size().unstack().plot()
plt.show()

output :