Print remaining lines in file after regular expression that includes variable

Question

I have the following data:

====> START LOG for Background Process: HRBkg Hello on 2013/09/27 23:20:20 Log Level 3
09/27 23:20:20 I Background process is using processing model #: 3
09/27 23:20:23 I
09/27 23:20:23 I -- Started Import for External Key
====> START LOG for Background Process: HRBkg Hello on 2013/09/30 07:31:07 Log Level 3
09/30 07:31:07 I Background process is using processing model #: 3
09/30 07:31:09 I
09/30 07:31:09 I -- Started Import for External Key

I need to extract the remaining file contents after the LAST match of ====> START LOG .....
I have tried numerous times to use sed / awk , however, I can not seem to get awk to utilize a variable in my regular expression. The variable I was trying to include was for the date (2013/09/30) since that is what makes the line unique.
I am on an HP-UX machine and can not use grep -A .

Any advice?

Answer 1

There's no need to test for a specific time just to find the last entry in the file:

awk '
    BEGIN { ARGV[ARGC] = ARGV[ARGC-1]; ARGC++ }
    NR == FNR { if (/START LOG/) lastMatch=NR; next }
    FNR == lastMatch { found=1 }
    found
' file

Answer 2

This might work for you (GNU sed):

a=2013/09/30
sed '\|START LOG.*'"$a"'|{h;d};H;$!d;x' file

Answer 3

This will return your desired output.

sed -n '/START LOG/h;/START LOG/!H;$!b;x;p' file

If you have tac available, you could easily do..

tac <file> | sed '/START LOG/q' | tac

Answer 4

Here is one in Python:

#!/usr/bin/python

import sys, re

for fn in sys.argv[1:]:
    with open(fn) as f:
        m=re.search(r'.*(^====> START LOG.*)',f.read(), re.S | re.M)
        if m:
            print m.group(1)

Then run:

$ ./re.py /tmp/log.txt
====> START LOG for Background Process: HRBkg Hello on 2013/09/30 07:31:07 Log Level 3
09/30 07:31:07 I Background process is using processing model #: 3
09/30 07:31:09 I
09/30 07:31:09 I -- Started Import for External Key

If you want to exclude the ====> START LOGS.. bit, change the regex to:

r'.*(?:^====> START LOG.*?$\\n)(.*)'

Answer 5

For the record, you can easily match a variable against a regular expression in Awk, or vice versa.

awk -v date='2013/09/30' '$0 ~ date {p=1} p' file

This sets p to 1 if the input line matches the date, and prints if p is non-zero.

(Recall that the general form in Awk is condition { actions } where the block of actions is optional; if omitted, the default action is to print the current input line.)

Answer 6

This prints the last START LOG , it set a flag for the last block and print it.

awk 'FNR==NR  { if ($0~/^====> START LOG/) f=NR;next} FNR>=f' file file

You can use a variable, but if you have another file with another date, you need to know the date in advance.

var="2013/09/30"
awk '$0~v && /^====> START LOG/ {f=1}f' v="$var" file
====> START LOG for Background Process: HRBkg Hello on 2013/09/30 07:31:07 Log Level 3
09/30 07:31:07 I Background process is using processing model #: 3
09/30 07:31:09 I
09/30 07:31:09 I -- Started Import for External Key

Answer 7

Answer in perl:

If your logs are in assume filelog.txt .

my @line;

open (LOG, "<filelog.txt") or "die could not open filelog.tx";

while(<LOG>) {
   @line = $_;
}

my $lengthline = $#line;
my @newarray;
my $j=0;

for(my $i= $lengthline ; $i >= 0 ; $i++) {
  @newarray[$j] = $line[$i];
  if($line[$i] =~ m/^====> START LOG.*/) {
    last;
  }
  $j++;
}

print "@newarray \n";

Answer 8

With GNU awk ( gawk ) or Mikes awk ( mawk ) you can set the record separator ( RS ) so that each record will contain a whole log message. So all you need to do is print the last one in the END block:

awk 'END { printf "%s", RS $0 }' RS='====> START LOG' infile

Output:

====> START LOG for Background Process: HRBkg Hello on 2013/09/30 07:31:07 Log Level 3
09/30 07:31:07 I Background process is using processing model #: 3
09/30 07:31:09 I
09/30 07:31:09 I -- Started Import for External Key