How to split csv rows containing multiple string values based on comma but without considering comma inside curly brackets { }

Question

I am reading one csv file and trying to split its rows based on comma, Here in my case rows containing some values having comma as a part of that value and that value start and ends with { }.

My Split function:

    def process(self, row):
        """
        Splits each row on commas
        """
        Uid, controlNo, profileType, LAStpointDetail, LastPointDate = 
                                                         row.split(",")

My row example:

0923,41003,Permanent,{""Details"": [{""data"": {""status"": ""FAILURE"", ""id"": ""12345""}, ""DetailType"": ""Existing""}]},2019-06-27

In rows If you see "LAStpointDetail", it already contains multiple commas in it. How do I split this whole row based on comma.

Answer 1

What you seem to have here is csv data, in which one column is encoded as json .

It's not possible to tell exactly how the data is quoted from the the question (it would have been better to paste the repr of the row), but let's assume it's something like this:

'"0923","41003","Permanent","{""Details"": [{""data"": {""status"": ""FAILURE"", ""id"": ""12345""}, ""DetailType"": ""Existing""}]}","2019-06-27"' 

If that's true, and you have a file of this data containing multiple rows, you can use the csv module to read it:

import csv
import json
with open('myfile.csv', 'rb') as f:
    reader = csv.reader(f)
    # Remove the next line if the first row is not the headers.
    # next(reader)    # Skip header row.
    for row in reader:
        Uid, controlNo, profileType, LAStpointDetail, LastPointDate = row
        # Load embedded json into a dictionary.
        detail_dict = json.loads(LAStpointDetail)
        # Do something with these values.

If you only have a single row as a string, you can still use the csv module:

>>> row = '"0923","41003","Permanent","{""Details"": [{""data"": {""status"": ""FAILURE"", ""id"": ""12345""}, ""DetailType"": ""Existing""}]}","2019-06-27"'
>>> # Make the data an element in a list ([]).
>>> reader = csv.reader([row])
>>> Uid, controlNo, profileType, LAStpointDetail, LastPointDate = next(reader)
>>> print Uid
0923
>>> d = json.loads(LAStpointDetail)
>>> d
{u'Details': [{u'DetailType': u'Existing', u'data': {u'status': u'FAILURE', u'id': u'12345'}}]}
>>> print d['Details'][0]['data']['status']
FAILURE