EmptyDataError：在字典中加载多个文件时没有要从文件中解析的列

Question

I have 1000 csv files that I call using the following code (which puts every file into a dictionary):

dataframes = {}
csv_root = Path(".")
for csv_path in csv_root.glob("*.csv"):
key = csv_path.stem
dataframes[key] = pd.read_csv(csv_path, skiprows=1)

However when I use this code I got this error

EmptyDataError: No columns to parse from file

Which indicates that there is empty data or header is encountered.

I would like to know how to identify which of those 1000 csv files are the ones causing troubles? Because, as you can understand, checking file by file will consume a lot of time.

Thanks a lot!

Answer 1

I would just use a try/catch, like so:

dataframes = {}
csv_root = Path(".")
for csv_path in csv_root.glob("*.csv"):
    key = csv_path.stem
    try:
        dataframes[key] = pd.read_csv(csv_path, skiprows=1)
    except Exception, as e:
        dataframes[key] = 'error' # mark the errored

This last step will get you the stems with issues:

errored_stems = [k for k,v in dataframes.iteritems() if k == 'error']