Pandas read_csv大文件将每列放入一个

Question

I have a csv file and I'm trying to read it via pandas command read_csv. However instead of reading 5 rows and 21 columns I get the following output:

There is a possible solution of manually changing the bad formatting of the file however, the real size is quite big(around 1GB) and it isnt possible to change it manually.

Answer 1

Consider below solution:

df = pd.read_csv("test.csv", sep="\"\,\"")
data = pd.concat([df.iloc[:,0].str.split(',', expand=True).rename(columns=dict(zip([0,1], df.columns[0].split(',')))), df.iloc[:,1:]], axis=1)

Would return a (5,21) shape of data .