Pandas 无法在一夜之间打开 csv 文件

Question

I am building a script that generates a csv file with pandas, but i'm trying to make it so if the file already exists in the designated path, the script will only append the new info into the existing file, while if the file doesn't exist, it will create a new one.

My code right now is like this:

#Trying to find the file in the designated path and then appending the new info

try:
   pd.read_csv('file.csv', encoding = 'ANSI')
   info.to_csv('file.csv', index = False, header = None, sep = ';', encoding = 'ANSI', decimal = ',', mode = 'a')

#Creating a new file into the existing path if it doesn't exist

except IOError:
   info.to_csv('file.csv',  index = False, sep = ';', encoding = 'ANSI', decimal = ',', header = True)

This works fine during the day, but when I try to run the script the next day, the pd.read_csv() encounters the following error:

ParserError: Error tokenizing data. C error: Expected 1 fields in line 10, saw 2

I've read that one way to resolve this is by adding the parameter error_bad_lines = False , but this results in an exponentially slower computational time. The verbose in this case shows that almost every line of the.csv file has some extra \n in it.

Is there some other way to tackle this file appending/creating problem?

I need to be able to open this file in excel to check the infos inside. Should I just try to create an.xlsx file instead of a.csv?

Answer 1

Use pathlib to check if the file already exists . If you really want to load the whole file (only to check if it exists) you would need to use the same parameters as in the pd.to_csv settings. But the latter is computationally very expensive for a simple check so I would recommend to use pathlib .

The default separator in pd.read_csv is , (which is your decimal separator) so the error is probably from trying to read the file with the wrong delimiter (more here ).

Possible solution:

from pathlib import Path

if Path('file.csv').exists():
    info.to_csv('file.csv', index = False, header = None, sep = ';', encoding = 'ANSI', decimal = ',', mode = 'a')
else:
    info.to_csv('file.csv',  index = False, sep = ';', encoding = 'ANSI', decimal = ',', header = True)