python＆pandas-将大型数据框拆分为多个数据框并绘制图表

Question

I'm under the similar condition with this case . I'm working on a project which has a large dataframe with about half-million of rows. And about 2000 of users are involving in this.( I get this number by value_counts() counting a column called NoUsager ).

I'd like to split the dataframe into several array/dataframe for plotting after. (Several means an array/dataframe for each user) I gott the list of users like:

df.sort_values(by='NoUsager',inplace=True)
df.set_index(keys=['NoUsager'],drop=False,inplace=True)
users = df['NoUsager'].unique().tolist()

I know what's after is a loop to generate the smaller dataframes but I have no idea how to make it happen. And I combined the code above and tried the one in the case but there was no solution for it.

What should I do with it?

---

EDIT

I want both histogram and boxplot of the dataframe. With the answer provided, I already have a boxplot of all NoUsager . But with large amount of data, the boxplot is too small to read. So I'd like to split the dataframe by NoUsager and plot them separately. Diagrams that I'd like to have:

1. boxplot, column= DureeService , by= NoUsager
2. boxplot, column= DureeService , by='Weekday`
3. histogram, for every Weekday ,by= DureeService

I hope this time is well explained.

DataType:

          Weekday NoUsager Periods  Sens  DureeService
DataType   string  string  string string datetime.time

Sample of DataFrame:

Weekday NoUsager Periods Sens DureeService
Lun 000001 Matin + 00:00:05 
Lun 000001 Matin + 00:00:04 
Mer 000001 Matin + 00:00:07 
Dim 000001 Soir  - 00:00:02 
Lun 000001 Matin + 00:00:07 
Jeu 000001 Soir  - 00:00:04 
Lun 000001 Matin + 00:00:07 
Lun 000001 Soir  - 00:00:04 
Dim 000001 Matin + 00:00:05 
Lun 000001 Matin + 00:00:03 
Mer 000001 Matin + 00:00:04 
Ven 000001 Soir  - 00:00:03 
Mar 000001 Matin + 00:00:03 
Lun 000001 Soir  - 00:00:04 
Lun 000001 Matin + 00:00:04 
Mer 000002 Soir  - 00:00:04 
Jeu 000003 Matin + 00:00:50 
Mer 000003 Soir  - 00:06:51 
Mer 000003 Soir  - 00:00:08 
Mer 000003 Soir  - 00:00:10 
Jeu 000003 Matin + 00:12:35 
Lun 000004 Matin + 00:00:05 
Dim 000004 Matin + 00:00:05 
Lun 000004 Matin + 00:00:05 
Lun 000004 Matin + 00:00:05 

---

And what bothers me is that none of these data is number, so each time they have to be converted.

Thanks in advance!

Answer 1

[g for _, g in df.groupby('NoUsager')] gives you a list of data frames where each dataframe contains one unique NoUsager . But I think what you need is something like:

for k, g in df.groupby('NoUsager'):
    g.plot(kind = ..., x = ..., y = ...) etc..

Answer 2

No need to sort first. You may try this with your original DataFrame:

# import third-party libraries
import pandas as pd
import numpy as np
# Define a function takes the database, and return a dictionary
def splitting_dataframe(df):
    d = {}                                   # Define an empty dictionary
    nousager = np.unique(df.NoUsager.values) # Getting the NoUsage list
    for NU in nousager:                      # Loop over NoUsage list
        d[NU] = df[df.NoUsager == NU]        # I guess this line is what you want most
    return d                                 # Return the dictionary
dictionary = splitting_dataframe(df)  # Calling the function

After this, you can call the DataFrame for specific NoUsager by:

dictionary[target_NoUsager]

Hope this helps.

---

EDIT

If you want to do a box plot, have you tried:

df.boxplot(column='DureeService', by='NoUsager')

directly? More information here: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.boxplot.html

---

EDIT

If you want a boxplot for several selected 'NoUsager':

targets = [some selected NoUsagers]
mask = np.sum([df.A.values == targets[i] for i in xrange(len(targets))], dtype=bool, axis=0)
df[mask].boxplot(column='DureeService', by='NoUsager')

If you want a histogram for a selected 'NoUsager':

df[target NoUsager].hist(column='DureeService')

If you still need to separate them, @Psidom's first line is good enough.