CSV文件中DoubleDouble引号中的逗号

Question

I have a string like given in the example:

data = '02 JAN 2014,FEB 2014,A,1.00,,,""1,235.100000"",""1,230.00"",Column'

how can I parse this string with using python's csv module?

data = StringIO.StringIO(data) 
reader = csv.reader(data, quoting=csv.QUOTE_NONE)

It separates the the string ""1,235.10000"" to two column: '""1' and '235.1000""'

how can I fix this and arrange the module to not to split commas if its in double-double quotes?

Answer 1

I'm not sure if this is good enough, but:

>>> import csv
>>> data = '02 JAN 2014,FEB 2014,A,1.00,,,""1,235.100000"",""1,230.00"",Column'
>>> reader = csv.reader([data.replace('""', '|')], quotechar='|')
>>> next(reader)
['02 JAN 2014', 'FEB 2014', 'A', '1.00', '', '', '1,235.100000', '1,230.00', 'Column']

You can keep with the StringIO or whatever, but passing in a list made the example code simpler :). If you actually have a file object, you could even just use a simple generator to transform the lines before you feed them to your reader:

def transform(file):
   for line in file:
       yield line.replace('""', '|')

with open('foo') as fin:
    reader = csv.reader(transform(fin), quotechar='|')
    ...

And transform can become as sophisticated as you like -- eg if you need to preserve the quotes for some reason.

Answer 2

The best way to handle this would be to repair your input file; two quote characters together at the start of a column are seen as a quoted empty value and removed from your input when normal quoting rules are in effect. The double quotes at the end are seen as part of the value.

You can repair the values after the fact by post-processing each row:

def rejoin_quoted(row):
    new_row = []
    it = iter(row)
    for col in it:
         new_row.append(col)
         if col.startswith('""'):
             new_col = [col]
             for col in it:
                 new_col.append(col)
                 if col.endswith('""'):
                     new_row[-1] = ','.join(new_col).strip('"')
                     break
    return new_row

Demo:

>>> row = ['02 JAN 2014', 'FEB 2014', 'A', '1.00', '', '', '""1', '235.100000""', '""1', '230.00""', 'Column']
>>> rejoin_quoted(row)
['02 JAN 2014', 'FEB 2014', 'A', '1.00', '', '', '1,235.100000', '1,230.00', 'Column']

Answer 3

One way of doing this would be to slightly modify your data in order to explicitly specify a quote-char and the escape character:

data = '02 JAN 2014,FEB 2014,A,1.00,,,"\"1,235.100000\"","\"1,230.00\"",Column'
parsed = csv.reader(data, delimiter=',', quotechar='"', escapechar='\\')