按熊猫数据框列的相同部分字符串分组

Question

I have several csv files and each one contains one stock price in one month and has millions of data. The raw csv data data is like:

AA_Candy.csv

Index   CompanyName      Time       Price
  1      AA Candy    030101090355   1.78
  2      AA Candy    030101091533   1.79
  .......
333498   AA Candy    031231145556   2.18

BB_Cookie.csv

   1     BB Cookie   030101090225   3.20
   2     BB Cookie   030101090845   3.14
  .......
391373   BB Cookie   031231145958   3.88

I use python and pandas to process the data, after I load and combine some of the datafiles, now I have a dataframe like:

frame:

Index   CompanyName      Time       Price
  1      AA Candy    030101090355   1.78
  2      AA Candy    030101091533   1.79
  .......
333498   AA Candy    031231145556   2.18
333499   BB Cookie   030101090225   3.20
333500   BB Cookie   030101090845   3.14
  .......
712871   BB Cookie   031231145958   3.88

The time 031231145958 represent 2013-12-31 14:59:58

now I want to get the highest price and final price in every one hour of each company, and get an output file like:

range_start   AA Candy/Max    AA Candy/Close    BB Cookie/Max     BB Cookie/Close
0301010900     1.79              1.77            3.20              3.10
........
0312311400     2.24              2.18            3.88              3.88

Therefore I want to groupby the CompanyName and first 8 character of Time to get the data of same company in one hour, then do the calculation to find the max price value and final price value of each company and output the outcome with same start hour in one row; let companyName/Max or Close be the new column name.

Because I am really new in pandas and dataframe, I have the following questions:

1. How to group the data by the first 8 character of Time Column(Object) and then get my expected value?
2. How to form a new output dataframe/matrix as my expected output?

Thanks!!

Answer 1

Perform a groupby on the company name and first 8 characters of your string timestamp (ie date plus hour). Then use agg on the price to get custom functions for each (first, max, min and last). Unstack the company names, swap the levels of the company names and open/high/low/close and optionally sort your symbols.

gb = (df.groupby(['CompanyName', df.Time.str[:8]])
        .Price
        .agg({'open': 'first', 
              'high': np.max, 
              'low': np.min, 
              'close': 'last'})[['open', 'high', 'low', 'close']]
        .unstack('CompanyName'))
gb.columns = gb.columns.swaplevel(0, 1)
>>> gb.sortlevel(level=0, axis=1)
CompanyName AA Candy                   BB Cookie                  
                open  high   low close      open  high   low close
Time                                                              
03010109        1.78  1.79  1.78  1.79      3.20  3.20  3.14  3.14
03123114        2.18  2.18  2.18  2.18      3.88  3.88  3.88  3.88

Answer 2

For your first question, you can use

df.groupby(df.Time.str[0:8])

For your second question, unstack should be what you want:

df.groupby(df.Time.str[0:8]).unstack()