Grouping contents in a text file in python

Question

I have an input file of the following format

CC   -----------------------------------------------------------------------
CC
CC   hgfsdh kjhsdt kjshdk
CC
CC   -----------------------------------------------------------------------
CC   Release of 18-Sep-2019
CC   -----------------------------------------------------------------------
CC
CC   Alex 
CC   -----------------------------------------------------------------------
CC   Copyrighted vvvncbm License
CC   -----------------------------------------------------------------------
//
ID   1.1
ED   text1
AN   text2.
CA   text3
CF   text4.
CC   -!- Some members 
CC       also on .
CC   -!- May be 
CC   -!- Re
PR   PRTSF; C000AS61;
DQ   Q6, AZW2_DANRE;  Q7, AZW2_DANRE;  Q97, AZW2_DONT;
DQ   Q8, AZW2_DONT;  Q9, AZW2_AZW2_DONT;  Q10, AZW2_CAFT;
//
ID   1.2
ED   text1
AN   text2.
CA   text3
CF   text4.
CC   -!- Some members 
CC       also on .
CC   -!- May be 
CC       second line
PR   PRTSF; DOC00;
DQ   Q6, AZW2_DANRE;  Q7, AZW2_DANRE;  Q97, AZW2_DONT;
DQ   Q8, AZW2_DONT;  Q9, AZW2_AZW2_DONT;  Q10, AZW2_CAFT;
DQ   Q15, AZW2_DANRE;  Q43, AZW2_DANRE;  Q049, AZW2_DONT;
//

I would like to group the data in this text file and store it in a json

I've tried the following,

import os
import json
from pprint import pprint

def text_to_json(f_input):
    location_data = []
    if os.path.exists(f_input):
        with open(f_input, 'r') as f:
            for line in f.readlines()[12:]:
                if line.strip() != '//' and line.strip() != '//' and line.strip():
                    print(line[:-1])

                pass
        # return json.dumps(data)


if __name__ == '__main__':
    f_input = 'input.txt'
    text_to_json(f_input)

I have skipped the first few lines with comments. if line.strip().= 'DELIMITER' and line.strip():= 'DELIMITER' and line.strip(): , the delimiter is // . However, I am not sure how to use \\ and group the data corresponding to each id.

I would like to group the data using delimiter and store the data of each id in json format.

{
'1.1' : 
{'DQ': {'Q6': AZW2_DANRE,  'Q7': 'AZW2_DANRE',  'Q97': 'AZW2_DONT'
'Q8': 'AZW2_DONT',  'Q9': 'AZW2_AZW2_DONT';  'Q10': 'AZW2_CAFT'},
'ED': 'text1',
'AN': 'text2.',
'CA': 'text3',
'CF': 'text4.',
'PR': 'PRTSF; C000AS61;',
'CC': ['Some members also on .', 'May be', 'Re']
 } 
'1.2' :
{'DQ': {'Q6': AZW2_DANRE,  'Q7': 'AZW2_DANRE',  'Q97': 'AZW2_DONT'
'Q8': 'AZW2_DONT',  'Q9': 'AZW2_AZW2_DONT';  'Q10': 'AZW2_CAFT',
'Q15': 'AZW2_DANRE',  'Q43': 'AZW2_DANRE',  'Q049': 'AZW2_DONT'},
'ED': 'text1',
'AN': 'text2.',
'CA': 'text3',
'CF': 'text4.',
'PR': 'PRTSF; DOC00;',
'CC': ['Some members also on .', 'May be second line']
}
}

I could create the above json by storing based on line numbers. However, the line number of each dataset keeps varying. For instance, the data stored against DQ has 2 lines in the first dataset and 3 in second. Any suggestions on how to proceed?

Answer 1

I would suggest taking the approach of constructing everything in memory, in dictionaries and arrays. In the code below everything is being accumulated into the d dictionary. And then dump the data from memory as a JSON object. It looks like you want to treat different types of lines differently ('CC' lines become an array, 'DQ' lines become a dictionary, and other lines just stored). So, here's how I would approach the code:

import os
import json
from pprint import pprint

def text_to_json(f_input):
    location_data = []
    if os.path.exists(f_input):
        with open(f_input, 'r') as f:

            # Accumulate all of the line data in this dictionary
            d = {}

            # This keeps track of the current ID, like 1.1 or 1.2
            current_line_id = ''

            for line in f.readlines()[12:]:
                if line.strip() != '//' and line.strip() != '//' and line.strip():
                    # print(line[:-1])
                    line_type = line[0:2]
                    line_data = line[5:-1]
                    if line_type == 'ID':
                        d[line_data] = dict()
                        current_line_id = line_data
                    elif line_type == 'CC':
                        if line_type not in d[current_line_id]:
                            d[current_line_id][line_type] = []
                        d[current_line_id][line_type].append(line_data)
                    elif line_type == 'DQ':
                        if line_type not in d[current_line_id]:
                            d[current_line_id][line_type] = {}
                        for dq in line_data.split(';'):
                            dq = dq.strip()
                            dq_key = dq[0:2]
                            dq_val = dq[4:]
                            if dq_key != '':
                                d[current_line_id][line_type][dq_key] = dq_val
                    else:
                        d[current_line_id][line_type] = line_data

                pass
            print(json.dumps(d, indent=2))
        # return json.dumps(data)


if __name__ == '__main__':
    f_input = 'input.txt'
    text_to_json(f_input)