json.dumps adding double quotes and backslash

Question

I'm converting a csv file with multiple records to a file with a json array. Solution is How to find last line in csv file when using DictReader . This works fine, except that resulting json in file has backslash (\\) and double-quotes; ie it's escaping the strings. For example:

{"Test Name": "Basic Tcp IPerf Test", " \\"Status\\"": "PASS", " \\"Test Start Time\\"": "20190103 08:07:41.662", " \\"Test End Time\\"": "20190103 08:08:44.051", " \\"Throughput\\"": "2095.2746", " \\"Jitter\\"": "", " \\"Packet Loss %\\"": "", " \\"Compute/VM\\"": "SendVm (xxxx)", " \\"Statistics Sampling Start Time\\"": "03 Jan 2019 08:07:44", " \\"Statistics Sampling End Time\\"": "03 Jan 2019 08:08:42", " \\"Min CPU\\"": "0", " \\"Max CPU\\"": "2", " \\"Avg CPU\\"": "1", " \\"Avg Memory (MB)\\" ": "7758"}

Is there a way to write out that json without that escaping? Code:

csvfile = open('fiile.csv','r')
jsonfile = open('file.json','w')

reader = csv.DictReader(csvfile)

jsonfile.write(json.dumps(list(reader)))

Answer 1

It looks like your csv file is formatted in a way that confuses the default csv parser. In particular, your columns are separated by both a comma and a space:

           comma
           v
"Test Name", "Status"
            ^
            space

This makes the parser assume that all the text following the comma is part of the value, including the space and the quote mark.

One possible solution is to specify a value for the skipinitialspace argument when opening the DictReader.

import csv
import json

csvfile = open('fiile.csv','r')
jsonfile = open('file.json','w')

reader = csv.DictReader(csvfile, skipinitialspace=True)
jsonfile.write(json.dumps(list(reader)))

Then the parser should understand that the space and quote aren't part of the value. Result:

[{"Test Name": "Basic Tcp IPerf Test", "Status": "PASS", "Test Start Time": "20190103 08:07:41.662", "Test End Time": "20190103 08:08:44.051", "Throughput": "2095.2746", "Jitter": "", "Packet Loss %": "", "Compute/VM": "SendVm (x.x.x.x)", "Statistics Sampling Start Time": "03 Jan 2019 08:07:44", "Statistics Sampling End Time": "03 Jan 2019 08:08:42", "Min CPU": "0", "Max CPU": "2", "Avg CPU": "1", "Avg Memory (MB)": "7758"}]