How to process each grep (with -A option) result separately? Not by line, but by item

Question

I have a long log to parse, the log message for each event takes more than one line usually. If I use hardcoded line numbers of each event, then I can use grep -A $EventLineNumbers to grab the whole message of each event.

Now, for example, I want to grep for two data fields in one event with 20 lines of messages. That event might be logged 100 times in the log. I want to avoid process grep results line by line because if one data field doesn't exist in one of the events, I will end up taking data from a different event and "bundle" it with previous event.

In short, how do I process each grep individually given that I have set a -A option in the command?

Answer 1

For this kind of parsing, I would use awk(1). Compared to grep(1), it allows you to easily use variables and conditions. So you can:

1. match on the line(s) indicating the start of a bunch of lines you are interested in
2. set a variable to remember this state
3. process lines following, remember relevant fields
4. when you reach the end of this bunch of lines, print the formatted information

If in fact, those lines are not interesting, you can reset the state and skip to the next interesting message.

The command will be longer than a simple grep, but it remains concise and you don't need to use a full blown programming language.

Answer 2

This should work:

Example Input:

1
2
3
1
4
5
6
2
4

Assuming that all blocks are 3 lines and that you want to match the block with a 1 in it. The following code works:

echo -e '1\n2\n3\n1\n4\n5\n6\n2\n4' | grep 1 -A2 | awk -v n=3 '{print} NR % 3 == 0 { printf "\0" }' | xargs -0 -n 1 echo -n "WOOT: "

Explanation:

The grep 1 -A2 is you expression to find the relevant block. I assume that the matching expression occurs only once per block. The -A2 is for the two lines of context after the match.

The awk -v '{print} NR % 3 == 0 { printf "\\0" }' part instructs awk to print a \\0 character every 3 lines. Again, adjust this for the relevant blocksize. We need this for the next command.

The xargs -0 -n1 part executes the next command (which would be the extra filters for you), giving the whole block to it. -0 for the null terminated items (a block) and -n1 for only passing a single item to the next command.

In my example this will give the following output:

WOOT:  1
2
3
WOOT:  1
4
5

Meaning that echo was executed for each block exactly once.