shift a column in a pandas dataframe will set data to NaN

Question

I try to shift a column in a dataframe, before I found shift method, I try to do it myself. I found something intersting.

Here is the code

create the dataframe:

df = pd.DataFrame(np.random.randn(5,3))

Here is the data:

          0         1         2
0 -0.274583  0.141030  1.335455
1 -0.887131 -1.253900  0.395358
2 -0.194082 -0.462745 -0.237422
3  0.095535  0.236268 -1.347387
4  2.210129 -0.816895 -1.225560

if I do this to shift the last column up 2 rows:

df.loc[0:2,2] = df.loc[2:,2]

The result is, the first 2 rows are NaN

          0         1         2
0 -0.274583  0.141030       NaN
1 -0.887131 -1.253900       NaN
2 -0.194082 -0.462745 -0.237422
3  0.095535  0.236268 -1.347387
4  2.210129 -0.816895 -1.225560

What is the problem here?

Answer 1

It seems need convert output to numpy array , for prevent align by index:

df.loc[0:2,2] = df.loc[2:,2].values
print (df)
          0         1         2
0 -0.274583  0.141030 -0.237422
1 -0.887131 -1.253900 -1.347387
2 -0.194082 -0.462745 -1.225560
3  0.095535  0.236268 -1.347387
4  2.210129 -0.816895 -1.225560

Explanation why your solution does not work:

Output is aligned by index, so if 0,1 indices dont exist add NaN s:

print (df.loc[2:,2])
2   -0.237422
3   -1.347387
4   -1.225560
Name: 2, dtype: float64