Extract into a new dataframe the last N positive value for each column

Question

I would like to extract the last N values > 0 for each column, and save them in a new dataframe. Could you suggest me an effective way to do it? Thank you!

Answer 1

##df   

   A
0  1
1  2
2  3
3  0
4 -1
5  3
6  4
7 -5

N = 2
df[df['A'] > 0].tail(N)

---

   A
5  3
6  4

Answer 2

Say you have a data frame like this and N is 3:

    0   1   2
0   1   -5  2
1   8   -5  -5
2   1   5   9
3   6   7   -6
4   -3  -6  -1
5   -7  1   9
6   -8  8   0
7   -8  3   1
8   -7  -9  7
9   5   5   -5
10  5   4   8
11  -4  -4  -2
12  -3  8   6
13  -1  1   -4
14  -3  9   4
15  -5  1   -4
16  3   6   -9
17  -4  -9  4
18  9   7   1
19  -1  -2  9

---

df_out = pd.DataFrame(columns=df.columns)
for col in df.columns:
    df_out[col] = df[df > 0][col].dropna().tail(3).reset_index(drop=True)

will give you the following output

---

    0   1   2
0   5.0 1.0 4.0
1   3.0 6.0 1.0
2   9.0 7.0 9.0

Answer 3

Use dictionary comprehension with filtering by mask with Series.gt and Series.tail , create default index values by Series.reset_index with drop=True and last pass to DataFrame constructor:

np.random.seed(2020)
df = pd.DataFrame(np.random.randint(-5, 5, size=(10, 3)))
print (df)
   0  1  2
0 -5  3 -2
1  1 -2 -2
2  2  3 -5
3 -5  3  4
4 -2  2 -3
5 -2  1  0
6 -5 -1  3
7  1 -1 -4
8 -4  0  4
9  0  1  1

d = {c: df.loc[df[c].gt(0), c].tail(3).reset_index(drop=True) for c in df.columns}
df = pd.DataFrame(d)
print (df)
   0  1  2
0  1  2  3
1  2  1  4
2  1  1  1