How to loop over multiple Pandas DataFrames using the filter function?

Question

So I have multiple data frames that I am attempting to loop over.

I have created a list using the following code:

data_list = [df1, df2, df3]

After that I would like to filter out a predefined range of numbers in the column 'Firm_Code' in each data frame.

So far, I am able to filter out firms with a respective code between 6000 and 6999 for a single data frame as follows:

FFirms = range(6000,7000)
Non_FFirms = [b for b in df1['Firm_Code'] if b not in FFirms]
df1 = df1.loc[df1['Firm_Code'].isin(Non_FFirms)] 

Now I would like to loop over the data_list . My first try looks like the following:

for i in data_list:
     i = i.loc[i.Firm_Code.isin(Non_FFirms)]

Appreciate any suggestions!

Answer 1

Instead of making the list of dataframes, you can concat all the data frames into a single dataframe.

data_df = pd.concat([df1,df2,df3],ignore_index=True)

In case you need identification from which dataframe you have fetched the value you can add a new column say 'Df_number'.

using data_df you can you can filter the data

FFirms = range(6000,7000)
Non_FFirms = [b for b in df1['Firm_Code'] if b not in FFirms]
filtered_data_df = data_df.loc[data_df['Firm_Code'].isin(Non_FFirms)]