在python中将文本片段转换为字典键值对/ JSON

Question

I am trying to convert the output of Prowler in the following format to a dictionary, and then converting the dictionary into a JSON file.

 0.1  Generating AWS IAM Credential Report... 

 1  Identity and Access Management **************************************** 

 1.1  Avoid the use of the root account (Scored).
       INFO! Root account last accessed (password key_1 key_2): 1970-01-01 00:00:00 N/A N/A 

 1.2  Ensure multi-factor authentication (MFA) is enabled for all IAM users that have a console password (Scored)
       WARNING! User XXXX has Password enabled but MFA disabled 
       WARNING! User XXXX has Password enabled but MFA disabled 

 1.3  Ensure credentials unused for 90 days or greater are disabled (Scored)
       WARNING! User "XXXX" has not logged in during the last 90 days  
       WARNING! User "XXXX" has not logged in during the last 90 days   
       OK!  User "XXXX" found with credentials used in the last 90 days
       OK!  User "XXXX" found with credentials used in the last 90 days

 1.4  Ensure access keys are rotated every 90 days or less (Scored)
       WARNING!  XXXXXXX has not rotated access key1 in over 90 days   

 1.5  Ensure IAM password policy requires at least one uppercase letter (Scored)
       OK!  Password Policy requires upper case

 1.6  Ensure IAM password policy require at least one lowercase letter (Scored)
       OK!  Password Policy requires lower case

In python, I have this function to parse the prowler.txt file, which uses regex to find the section header as a key value for the dictionary, and then parse the text file after a header match to add the lines underneath as the value for the key.

def create_master_report(ec2_info):
    prowler_file = 'reports/prowler.txt'
    findings = {}
    with open(prowler_file, 'r') as f:
        for line in f:
            if re.search('\s\d\.\d\d*\s\s\w', line):
                header = line.strip()
                findings.update({header: []})
    for i in findings:
        prowler_findings = []
        with open(prowler_file, 'r') as f:
            for index, line in enumerate(f, start=1):
                if line.strip() == i:
                    for line in enumerate(f, start=index+1):
                        if line != r'\\n':
                            #if re.search('WARNING!', line):
                            prowler_findings.append(str(line).strip())
                        if line == r'\\n':
                            break
        findings.update({i: prowler_findings})
    report_json['Prowler Results'].update(findings)
    with open(master_report, 'w') as outfile:
        json.dump(report_json, outfile, sort_keys=True)

However, I seem to be looping through the entire document and adding much more than anticipated as the key value. The end goal here is to parse the document starting at the line after the header, and then break once a new line is detected. I think a while loop would work, but I can't seem to implement one that loops through each line and breaks on a new line. In addition, I would only want to pull in lines that contain 'WARNING!' , but I have that commented out in order to test the basic functionality.

Can anyone provide any insight on how to do this?

Answer 1

You can do this in one loop. For each line, if it is a header, update the current key, else append the line in the findings with current key:

findings = {}
key = None
with open('reports/prowler.txt') as f:
   for line in f:
       if re.search(r'\s\d\.\d\d*\s\s\w', line):
          key = line.strip()
          findings[key] = []
       elif key is not None:
           findings[key].append(line)

Answer 2

There is a good answer yet. But note that:

if line == r'\\n':

does not work if you want to match (or not) a single carriage return. It should be simply:

if line == '\n':