New Pandas column with cumulative value depending on condition on the previous row

Question

I have the following dataframe which consist of three columns: values, cumsum, ema.

import pandas as pd
import numpy as np

col_values = [-36.0,
 1.5,
 -13.200000000000273,
 32.79999999999973,
 11.69999999999959,
 31.59999999999991,
 52.0,
 -1.1999999999995907,
 52.0,
 -105.29999999999973,
 -3.299999999999727,
 -14.100000000000136,
 9.599999999999454,
 -90.89999999999986,
 0.0,
 33.40000000000009,
 -12.900000000000546,
 39.09999999999991,
 32.80000000000018,
 15.299999999999727,
 -33.0,
 -15.599999999999454,
 11.700000000000273,
 12.600000000000136,
 -6.0]

col_cumsum = [1717.6999999999982,
 1719.1999999999982,
 1705.999999999998,
 1738.7999999999977,
 1750.4999999999973,
 1782.0999999999972,
 1834.0999999999972,
 1832.8999999999976,
 1884.8999999999976,
 1779.5999999999979,
 1776.2999999999981,
 1762.199999999998,
 1771.7999999999975,
 1680.8999999999976,
 1680.8999999999976,
 1714.2999999999977,
 1701.3999999999971,
 1740.499999999997,
 1773.2999999999972,
 1788.599999999997,
 1755.599999999997,
 1739.9999999999975,
 1751.6999999999978,
 1764.299999999998,
 1758.299999999998]

col_ema = [1754.3539071732214,
 1751.0059160138665,
 1746.71963829826,
 1745.965387031759,
 1746.3972549334958,
 1749.7975163684007,
 1757.8263243333147,
 1764.9761982063321,
 1776.3975126628718,
 1776.702511456884,
 1776.6641770324188,
 1775.2866363626642,
 1774.9545757566962,
 1765.9969971132011,
 1757.892521197658,
 1753.7408525121668,
 1748.75600941577,
 1747.969722804744,
 1750.3821301566725,
 1754.0219272846084,
 1754.1722199241694,
 1752.8224846932958,
 1752.7155813891723,
 1753.8188593521081,
 1754.2456346519073]


df1 = pd.DataFrame({'values': col_values, 'cumsum': col_cumsum, 'ema': col_ema})
df1.head(50)

I want to create a new column which adds cumulative 'values' column if in the previous row the value of 'cumsum' is equal or greater than 'ema'.

I have tried this:

df1['equity_cond'] = np.where(df1['cumsum'].shift(1)>=df1['ema'].shift(1), df1['equity_cond'].shift(1) + df1['values'], df1['equity_cond'].shift(1))

Which generates the following error:

KeyError: 'equity_cond'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-25-1c89c1e1113c> in <module>
      ----> 2 df1['equity_cond'] = np.where(df1['cumsum'].shift(1)>=df1['ema'].shift(1), df1['equity_cond'].shift(1) + df1['values'], df1['equity_cond'].shift(1))

How this can be implemented in Pandas?

---

Edited:

Expected values for this new column would be:

0
0
0
0
0
31.6
83.6
…

Answer 1

You can multiply your ['values'] column by the condition, so that when you apply .cumsum() , it adds 0 for the rows where your condition is not met, and the value of the ['values'] column when it is met.

cond = df1['cumsum'].shift() >= df1['ema'].shift()
df1['x'] = np.where(cond,
                   (df1['values'] * cond).cumsum(),
                   0)

This appears to give the result you are looking for:

    values  cumsum          ema      x
0    -36.0  1717.7  1754.353907    0.0
1      1.5  1719.2  1751.005916    0.0
2    -13.2  1706.0  1746.719638    0.0
3     32.8  1738.8  1745.965387    0.0
4     11.7  1750.5  1746.397255    0.0
5     31.6  1782.1  1749.797516   31.6
6     52.0  1834.1  1757.826324   83.6
7     -1.2  1832.9  1764.976198   82.4
8     52.0  1884.9  1776.397513  134.4
9   -105.3  1779.6  1776.702511   29.1
10    -3.3  1776.3  1776.664177   25.8

Answer 2

Two ways:

1. Using Pandas and similar to what is suggested by Brendan.

cond = df1['cumsum'].shift() >= df1['ema'].shift()
df1['equity_cond'] = (df1['values'] * cond).cumsum()

---

2. Or do it like so then add the column to df1:

 # your previous code goes here equity = [0 for x in range(len(df1))] for i, n in enumerate(equity): if i==0: continue if df1["cumsum"][i-1] >= df1["ema"][i-1]: equity[i] = equity[i-1] + df1["values"][i] else: equity[i] = equity[i-1] df1["equity_cond"] = equity