我可以在分组的数据帧上应用使用“移位”的函数，并从熊猫返回一个简单的数据帧吗？

Question

I hope the subject line is relatively clear. I'm using python/ pandas, and I'm working with daily pricing data on equities. I have one large csv file with data on 4000+ symbols, with approximately 100 days' data. So there are many repeated date and symbol values, but symbol/ date combinations are unique. I'm trying to get percentage change on each ticker/ date combination, for multiple lag (shift) dates. On a dataset of one symbol, this would be as simple as

(dataframe.Close - dataframe.Close.shift(1)) / dataframe.shift(1).

Here is a sample of the initial data:

TradeDate    Symbol    Close
1/1/2014     A         10.00
1/2/2014     A         11.00
1/3/2014     A         10.50
1/1/2014     B         2.00
1/2/2014     B         2.10
1/3/2014     B         2.05

The output I'm trying to get is:

TradeDate    Symbol    Perf1    Perf2
1/1/2014     A         NA       NA
1/2/2014     A         0.10     NA
1/3/2014     A         -0.045   0.05
1/1/2014     B         NA       NA
1/2/2014     B         0.05     NA
1/3/2014     B         -0.024   0.025

I'm new to pandas, and I've been scouring the web for a similar example or more general treatment on applying vectorized functions on groups in pandas. I'm not having much luck; I experimented with more traditional methods, looping over a list of unique tickers, calculating the performance percentages individually, assembling them into a data frame, then appending that to a 'master' data frame. It works, but takes 20+ minutes (and happens to be extremely clunky). I'm sure there's a better way. But I don't yet know enough of how to ask for specific functionality details.

Can anyone help? Thanks...

Answer 1

I think you can use groupby and pct_change (don't blame me for the name..).

First, let's make sure everything's a real time and sort it:

>>> df["TradeDate"] = pd.to_datetime(df["TradeDate"])
>>> df = df.sort(["Symbol", "TradeDate"])
>>> df
   TradeDate Symbol  Close
0 2014-01-01      A  10.00
1 2014-01-02      A  11.00
2 2014-01-03      A  10.50
3 2014-01-01      B   2.00
4 2014-01-02      B   2.10
5 2014-01-03      B   2.05

And then do the work:

>>> df.groupby("Symbol")["Close"].pct_change()
0         NaN
1    0.100000
2   -0.045455
3         NaN
4    0.050000
5   -0.023810
dtype: float64
>>> df["Perf1"] = df.groupby("Symbol")["Close"].pct_change()
>>> df["Perf2"] = df.groupby("Symbol")["Close"].pct_change(2)
>>> df
   TradeDate Symbol  Close     Perf1  Perf2
0 2014-01-01      A  10.00       NaN    NaN
1 2014-01-02      A  11.00  0.100000    NaN
2 2014-01-03      A  10.50 -0.045455  0.050
3 2014-01-01      B   2.00       NaN    NaN
4 2014-01-02      B   2.10  0.050000    NaN
5 2014-01-03      B   2.05 -0.023810  0.025

It would probably be cleaner to do the grouping once, eg

grouped = df.groupby("Symbol")["Close"]
for i in range(1,5):
    df["Perf{}".format(i)] = grouped.pct_change(i)

or something.