Extract pandas dataframe column names from query string

Question

I have a dataset with a lot of fields, so I don't want to load all of it into a pd.DataFrame , but just the basic ones.

Sometimes, I would like to do some filtering upon loading and I would like to apply the filter via the query or eval methods, which means that I need a query string in the form of, ie "PROBABILITY > 10 and DISTANCE <= 50" , but these columns need to be loaded in the dataframe.

Is is possible to extract the column names from the query string in order to load them from the dataset?

I know some magic using regex is possible, but I'm sure that it would break sooner or later, as the conditions get complicated.

So, I'm asking if there is a native pandas way to extract the column names from the query string.

Answer 1

I think you can use when you load your dataframe the term use cols I use it when I load a csv I dont know that is possible when you use a SQL or other format.

Columns_to use=['Column1','Column3'] pd.read_csv(use_cols=Columns_to_use,...)

Thank you