Extract specific lines from a text

Question

I want to extract the warnings from this text that are related to all the files that are obj_x.o, but they are right at the end of the lines and i have to check them, but then, i cannot print the next lines with the warnings until the code find the "[" character.

----Running------

[458/1060] Building C object Windows/APPL/src/obj_a.o

[459/1060] Building C object Windows/APPL/src/obj_abc.o

In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u

        ^

D:/Windows/APPL/src/obj_abc.o note: previous definition is here

#define DCM_PROGRAMMING_SESSION                0x02U


       ^

1 warning generated.

[460/1060] Building C object Windows/APPL/src/obj_b.o

[461/1060] Building C object Windows/APPL/src/obj_c.o


10123 errors and 582 warnings occured

Current work dir changed to: ...

----End------

this is the file. And this is my try:

obj = ["obj_a.o", "obj_abc.o", "obj_b.o", "obj_c.o"]

with open("build.txt", "rt") as myfile:
    line = myfile.readlines()
    for line in myfile:
        for i in range(len(obj)):
            if obj[i] in line:
                for j in range(i, len(myfile)):
                    if not line.startswith("["):
                        print(line)

This must be the result.

In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u

        ^

D:/Windows/APPL/src/obj_abc.o note: previous definition is here

#define DCM_PROGRAMMING_SESSION                0x02U


       ^

1 warning generated.

Answer 1

I think that you can also use python ttp module for that purpose. Please see the solution using this module. The code should work if your data has similar templates. I will put comments to explain each step of what I have done.

I have added more data to see if it works fine. Please kindly see the updated version of your data along with first part of the code:

from ttp import ttp
import json

data_to_parse = """
----Running------

[458/1060] Building C object Windows/APPL/src/obj_a.o

[459/1060] Building C object Windows/APPL/src/obj_abc.o

In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u

        ^

D:/Windows/APPL/src/obj_abc.o note: previous definition is here

#define DCM_PROGRAMMING_SESSION                0x02U


       ^

1 warning generated.

[460/1060] Building C object Windows/APPL/src/obj_b.o

[461/1060] Building C object Windows/APPL/src/obj_c.o


10123 errors and 582 warnings occured

Current work dir changed to: ...

[468/1060] Building C object Windows/APPL/src/obj_axyz.o

In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u

        ^

D:/Windows/APPL/src/obj_abc.o note: previous definition is here

#define DCM_PROGRAMMING_SESSION                0x02U


       ^

2 warning generated.

[460/1060] Building C object Windows/APPL/src/obj_b.o

10123 errors and 582 warnings occured

Current work dir changed to: ...

[487/1060] Building C object Windows/APPL/src/obj_bxyz.o

In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u


       ^

3 warning generated.

[460/1060] Building C object Windows/APPL/src/obj_b.o

----End------
"""

Please see the ttp template used to parse your data. The parsed data will be also converted to .json format.

ttp_template = """
[{{ignore}}] {{ignore}} {{ignore}} {{ignore}} {{ignore}}obj_{{Obj_x}}.o
{{Warning_Number|DIGIT}} warning generated.
"""
parser = ttp(data=data_to_parse, template=ttp_template)
parser.parse()

results = parser.result(format='json')[0]

result = json.loads(results)

See the print(result) output as .json format.

As the len of desired data is 2, the following part of code is used to create a new list which consists of only required data.

line_error_couples = []

for i in result[0]:
    if len(i) == 2: # Which means that the code hit the Warning. 
        line_error_couples.append(i)

See the output of line_error_couples consists of only desired data:

The indexes of where desired obj_x and warnings have been found and put into a dict file called obj_warn_indices_couples .

lines = data_to_parse.splitlines()

obj_indices = [] # is to find out indexes of "[468/1060] Building C object Windows/APPL/src/obj_x"
warning_indices = [] # is to find out indexes of "_x_ warning generated."

for obj in line_error_couples:
    lines_error_obj = f"object Windows/APPL/src/obj_{obj['Obj_x']}.o" # I have used "object" to capture the related line.
    lines_error_warning = f"{obj['Warning_Number']} warning generated"

    for id, line in enumerate(lines):
        if lines_error_obj in line:
            obj_indices.append(id)
        if lines_error_warning in line:
            warning_indices.append(id)

obj_warn_indices_couples = dict(zip(obj_indices, warning_indices)) # Then, both lists are converted to a dict file. 

See the output of the dict file:

The last part of the code is where desired data is printed. You can also put this data into a .txt file if it needs to be saved.

for k,v in obj_warn_indices_couples.items():
    print("\n################ See Warnings ##############\n")
    for line in lines[k+1:v+1]: # This prints only desired data, where warnings observed, "between obj_x" and "warnings generated."
        print(line)

Please see the desired output after the code is run:

################ See Warnings ##############


In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u

        ^

D:/Windows/APPL/src/obj_abc.o note: previous definition is here

#define DCM_PROGRAMMING_SESSION                0x02U


       ^

1 warning generated.

################ See Warnings ##############


In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u

        ^

D:/Windows/APPL/src/obj_abc.o note: previous definition is here

#define DCM_PROGRAMMING_SESSION                0x02U


       ^

2 warning generated.

################ See Warnings ##############


In file included from D:/Windows/APPL/src/obj_abc.o

D:/Windows/APPL/src/obj_abc.o warning: Not good

#define DCM_PROGRAMMING_SESSION 0x02u


       ^

3 warning generated.