Apply function in pandas using os.path module running slow

Question

I'm using .apply on a DataFrame column ( Path ) to determine whether values represent a file or folder:

dir_tree_df['Type'] = dir_tree_df['Path'].apply(
        lambda row: 'File' if os.path.isfile(row) else 'Folder')

I'm finding the above to be very slow. Is this specifically to do with os.path.isfile or just user defined functions in general, and if so, is there a more efficient way of achieving my goal?

Answer 1

I'm not sure how much fast this is, but this should get you what you want and use a function which is generally faster than lambda

I also just made up some data so you might have to play with it a little to reflect your own df.

def file_folder(x):
    if os.path.isfile(x) == True:
        return 'File'
    else:
        return 'Folder'

data = {
    'Column1' : [a file, a folder]
}
df = pd.DataFrame(data)
df['Type'] = df.apply(lambda x : file_folder(x['Column1']), axis = 1)
df