How to create function with help of pandas to read all file in a folder and create different dataframe for each file?

Question

I am trying to create a python function which help to do following Tasks

• Read.csv files from a folder
• Create different data-frame for each file with (dataframe name should be same as file name)
• Create list of all created data frame and assign same to a variable(variable name is name of folder)

Below is the Code I am trying:

import pandas as pd
import os

def read_folder():
    path = input('Please provide path name to read:')
    for file in range(1000):
        if os.path.exists(path + '/' + str(file) + '.csv'):
            file = pd.read_csv(path + '/' + str(file) + '.csv')
            folderpath = (os.path.split(path)[1])
            temp = []
            temp.append(file)
            print(temp)
        else:
            print('No file at given location')

I have also tried different answers available in this site but somehow most of those have different goal. I am running above code for it doesn't work for me.

Did I miss something on the above code?

Answer 1

if you wnat to keep all dataframes with its names then first you should create dictionary instead of list, and second you should create it before for -loop. If you create temp inside for -loop then you create it again and again and you remove previous content - so finally you have only last dataframe in temp

And when you will have dictionary with then you can get its keys to have all filenames.

BTW: It is good to use input() outside function and send path as argument - this way you can test it also with path from file or sys.argv or hardcoded name.

---

import pandas as pd
import os

# --- functions ---

def read_folder(path, min_number=0, max_number=1000):
    
    all_dfs = dict()
    
    for number in range(min_number, max_number):
        
        filename = f'{number}.csv'
        fullpath = os.path.join(path, filename)
        
        if os.path.exists(fullpath):
            all_dfs[filename] = pd.read_csv(fullpath)
        else:
            print('No file at given location')
    
    return all_dfs
            
# --- main ---

all_folders = dict()  # dictionary for all folders and filenames

path = input('Please provide path name to read:')

all_dfs = read_folder(path)
all_filenames = list(all_dfs.keys())

folder = os.path.split(path)[-1]
all_folders[folder] = all_filenames