Pandas: modify multiple dataframes (in a loop)

Question

I have multiple data frames that I want to do the same function for them. therefore I need to iterate over my frameworks.

# read text files 
df1 = pd.read_csv("df1.txt", sep="\t", error_bad_lines=False, index_col =None)
df2 = pd.read_csv("df2.txt", sep="\t", error_bad_lines=False, index_col =None)
df3 = pd.read_csv("df3.txt", sep="\t", error_bad_lines=False, index_col =None)

I have used the following code, however, it is not working (it means that all dataframes are still the same, and the changes do not affect them):

for df in [df1 , df2 , df3]:
    df = df[df["Time"]>= 600.0].reset_index(drop=True)
    df.head()

How I can iterate over them? and how can I overwrite dataframes?

Answer 1

The problem is that you're not changing the data frames in place, but rather creating new ones. Here's a piece of code that changes things in-place. I don't have your data, so I create fake data for the sake of this example:

df1 = pd.DataFrame(range(10))
df2 = pd.DataFrame(range(20))
df3 = pd.DataFrame(range(30))
df_list = [df1, df2, df3]

for df in df_list: 
    # use whatever condition you need in the following line
    # for example, df.drop(df[df["Time"] < 600].index, inplace=True)
    # in your case. 
    df.drop(df[df[0] % 2 == 0].index, inplace=True)
    df.reset_index(inplace = True)

print(df2) # for example

The result for df2 is:

   index   0
0      1   1
1      3   3
2      5   5
3      7   7
4      9   9
5     11  11
6     13  13
7     15  15
8     17  17
9     19  19

Answer 2

This might work:

df_list=[df1,df2,df3]

for df in range(len(df_list)):
    df=df_list[i]
    df_list[i]=df[df["Time"]>=600.0].reset_iundex(drop=True)

Answer 3

If you just store the new df to another list or same list you are all good.

newdf_list = []                                 # create new list to store df
for df in [df1 , df2 , df3]:
    df = df[df["Time"]>= 600.0].reset_index(drop=True)
    df.head()
    newdf_list.append(df)                       # append changed df to new list