CSV: when unicode is not unicode?

Question

Why writerow() says I'm passing str , when I'm passing unicode ?

import io
import csv

with io.open('test.csv', 'w', encoding="utf-8") as f:
    writer = csv.writer(f)
    x = [unicode(v, 'utf8') for v in ['id:ID', 'pos:string', 'definition:string', ':LABEL']]
    print x
    print type(x[0])
    writer.writerow(x)

---

[u'id:ID', u'pos:string', u'definition:string', u':LABEL']
<type 'unicode'>
Traceback (most recent call last):
 File "testcsv.py", line 9, in <module>
     writer.writerow(x)
  TypeError: write() argument 1 must be unicode, not str

Answer 1

The csv module in python 2 does not handle unicode . The python 2 documentation gives an example of how to create your own unicode csv handlers instead.

A better option would be to install the backports.csv module, which allows your python 2 code to use the newer python 3 csv api, which handles unicode.

After installing the library using pip install backports.csv , this code works in python 2:

>>> import io
>>> from backports import csv
>>> with io.open('test.csv', 'w', encoding="utf-8") as f:
>>>     writer = csv.writer(f)
>>>     x = [unicode(v, 'utf8') for v in ['id:ID', 'pos:string', 'definition:string', ':LABEL']]
>>>     print x
>>>     print type(x[0])
>>>     writer.writerow(x)
[u'id:ID', u'pos:string', u'definition:string', u':LABEL']
<type 'unicode'>

>>> with io.open('test.csv', encoding="utf-8") as f:
>>>     print f.read()
id:ID,pos:string,definition:string,:LABEL