简体   繁体   中英

Need to parse a log file against a Blacklist

Afternoon Experts.

I have a Log file that is populated with Errors. Here is a Condensed Version :

*****2015-05-19 17:10:23,536 [ajp-0.0.0.0-8009-31] (TqlBuilder.java:347) ERROR - DUMMY From TqlBuilder getQueryDefinition
java.lang.Exception
xxxxxxx
xxxxxxxxx
xxxxxxx
2015-05-19 17:10:23,588 [ajp-0.0.0.0-8009-31] (TqlBuilder.java:348) ERROR - DUMMY From SqlBuilder 
java.lang.Exception
xxxx
xxxx
xxxx
xxxxxxxx

2015-05-19 17:10:23,598 [ajp-0.0.0.0-8009-31] (TqlBuilder.java:351) ERROR - Bla Bla Bla java.lang.Exception xxxx xxxx xxxx xxxxxxxx**

I would like to come up with a Bash-Based Solution (been trying with grep, but with limited success) to pass this Log through a Blacklist File populated with (list of examples errors below), to clean the original file of these errors and produce a new file with all the errors from the blacklist cleared, along with the Lines that follow.

Here is an Example of my Blacklist

(TqlBuilder.java:347) ERROR - DUMMY From TqlBuilder getQueryDefinition
(TqlBuilder.java:348) ERROR - DUMMY From SqlBuilder 

Using plain grep I was able to remove just the lines, but I need a Solution to remove the lines that follow the Error. I thought of having grep remove all lines between the Date 'Tags' but I was not able to figure out how to go about that.

My Expected output for the new, cleaned file after applying the original against the blacklist, would be:

**2015-05-19 17:10:23,598 [ajp-0.0.0.0-8009-31] (TqlBuilder.java:351) ERROR - Bla Bla Bla
    java.lang.Exception
    xxxx
    xxxx
    xxxx
    xxxxxxxx**

Grep will not be able to do this by itself. You will need to either use a compiled program or script of some sort (language not important). While you could do this using multi-line matching regexes. It would be more straight forward to track state when walking through the file. In pseudo code something like:

inBlackList = false
while ( read next line )
{
    if ( ( inBlackList == false ) && ( current line starts blacklisted entry ) )
    {
        inBlackList = true
    }
    else if ( ( inBlackList == true ) && 
              ( current line starts new error ) && ( current line does not start blacklisted entry ) )
    {
        inBlackList = false
    }

    if ( inBlackList == false )
    {
        output current line to new file
    }
}  #  End while

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM