UnicodeEncodeError with Python 2.7 csv.writer

Question

I am trying to convert data from a json file to a csv file.

    with open('data.json') as data_file:    
        x = json.load(data_file)

    g = csv.writer(open("output.csv", "wb+"))
    g.writerow(["name", "price", "1 star", "2 star", "3 star", "4 star", "5 star", "review_author", "review_comment_count", "review_header", 
        "review_posted_date", "review_rating", "review_text", "url"].encode('utf-8'))

    for x in x:
        for i in range(len(x["reviews"])):
            g.writerow([x["name"],
                x["price"],
                x["ratings"]["1 star"],
                x["ratings"]["2 star"],
                x["ratings"]["3 star"],
                x["ratings"]["4 star"],
                x["ratings"]["5 star"],
                x["reviews"][i]["review_author"],
                x["reviews"][i]["review_comment_count"],
                x["reviews"][i]["review_header"],
                x["reviews"][i]["review_posted_date"],
                x["reviews"][i]["review_rating"],
                x["reviews"][i]["review_text"],
                x["url"]])

I expect to get a CSV file with all the columns (price, 1 star, 2 star, etc) with rows as corresponding data.

Instead I get an error:

Traceback (most recent call last): File "scraper5.py", line 162, in ReadAsin() File "scraper5.py", line 159, in ReadAsin x["url"].encode('utf8')]) UnicodeEncodeError: 'ascii' codec can't encode character u'\’' in position 377: ordinal not in range(128)

If I comment out line 159 I get the same error for the x["reviews"][i]["review_text"]. I've looked up previous questions on here but none of the solutions seem to work. Any suggestions for how to fix this?

Answer 1

Your script is failing because the json file has contained some non-ascii chars.And the system default encoding char-set ascii cannot trans it , which is out of range(128).

Try to explicit encode the all the lines your want to output into the file , like

g.writerow([x["name"].encode('utf8'),
            x["reviews"][i]["review_author"].encode('utf8'),
            x["reviews"][i]["review_comment_count"].encode('utf8')
            x["url"].encode('utf8')])