在有条件的情况下遍历熊猫数据框

Question

I am trying to iterate through a large data frame. However, I can't figure out how to include some conditions. Below is an example of my data frame:

       0        1        2    3
0  chr3R  4174822  4174922    1.0
1  chr3R  4175400  4175500    0.0
2  chr3R  4175466  4175566    0.5
3  chr3R  4175521  4175621    1.0
4  chr3R  4175603  4175703    0.0

I want to iterate through the rows and find the row where the difference of column 1 of x row with column 1 of row 1 is less than 5000. If the difference between x row and row 1 is less than 5000 then select the values of column 3 for rows x to 1 to put into a list. I then want to iterate this condition through out the data frame and make a list of lists for values of column 3.

I tried using iterrows() but I just go through the entire data frame and get nothing out.

Thanks.

Rodrigo

Answer 1

This can be done without using iterrows. All above will work. Another approach will be to use np.where from package numpy. This is an example. Pl. modify per your requirement.

    import numpy as np 
    df['newcol'] = np.where(df[1]- df[1].iloc[0] <  5000, 1, df[1])
    dfList = df['newcol'].tolist()

Answer 2

Let's take your dataset, adding a few more rows to that.

>>> import pandas as pd
>>> data = pd.DataFrame([
...     ['chr3R', 4174822, 4174922, 1.0],
...     ['chr3R', 4175400, 4175500, 0.0],
...     ['chr3R', 4175466, 4175566, 0.5],
...     ['chr3R', 4175521, 4175621, 1.0],
...     ['chr3R', 4175603, 4175703, 0.0],
...     ['chr3R', 5005603, 4175703, 0.0],   # col 1 is more than 5000 away
...     ['chr3R', 6005603, 4175703, 0.0],   # col 1 is more than 5000 away
... ])

The last two rows were added to show an example of a row where column 1 is more than 5000 higher than 4174822 (the first value of column 1.)

You can filter out those values which are within 5,000 of the first value of 4174822 as follows:

>>> subset = data[data[1] - data[1][0] < 5000]
>>> subset
       0        1        2    3
0  chr3R  4174822  4174922  1.0
1  chr3R  4175400  4175500  0.0
2  chr3R  4175466  4175566  0.5
3  chr3R  4175521  4175621  1.0
4  chr3R  4175603  4175703  0.0

... and then iterate using .iterrows() .

>>> for index, row in subset.iterrows():
...     # do something with row

Answer 3

>>> df[(df.iloc[:, 1] - df.iat[1, 1]) < 5000][3].tolist()
[1.0, 0.0, 0.5, 1.0, 0.0]

df.iloc[:, 1] selects all rows in column 1, subtracts the value at row one, column one using df.iat[1, 1] , and then filters for values less than 5000.

the [3] at the end then selects the third column (which would return a Series). But since you want a list, just append .tolist() to the result.

Answer 4

Great Thanks Guys,

However, I need to create a list of lists. I can grab the first rows that have a difference of 5000 with the first row. I need to grab the next rows with a difference of 5000. What is the best way to iterate through this process?

Thanks.