在带有多个双引号的字段内读取带有分隔符的 csv 文件

Question

I have a csv file with , separating the columns that I want to read with pandas, ie df = pd.read_csv("myfile.csv",sep=',',dtype=str,encoding='utf-8') . Columns are of various types, but I want to read everything as strings. One row of the following form causes the reader to see more columns in that row than it expected:

# column 1, column 2, column 3, ...
46745,"\\"\\"\\"blabla\\"\\" a, b bli\\"\\"more bla.\\"\\" bl blu \\"\\"bli bla blub\\"\\"\\"","something else",...

Some of the fields, as the second and third column here, are in double quotes. What sets the second field apart from other double quoted fields is that it contains several quotes such that when the parser reaches the , it does not realize that it is actually still within a quote which it should have recognized because the final quote character should be followed by the delimiter. Interestingly, when you pass engine='python' the parser actually recognized this, but instead of drawing the conclusion that the quote simply has not finished yet it throws the error ',' expected after '"' .

I tried all kinds of combinations for the keyword arguments quoting , quotechar , sep and engine . All to no avail.

Edit: Example as requested

import io
s = 'column1,column2,column3\n3463,hello,"more, stuff"\n46745,"\\"\\"\\"blabla\\"\\" a, b bli\\"\\"more bla.\\"\\" bl blu \\"\\"bli bla blub\\"\\"\\"","something else"'
df = pd.read_csv(io.StringIO(s),sep=',',dtype=str,encoding='utf-8')

If you comment out the last line, it works.

Answer 1

Usually quotes inside a field are escaped with another quote ( " ) which could be the default for the parser.

You probably need to use escapechar = '\\' in this case.