Extract two substrings from line in bash then concatenate them

Question

This solution will be part of a script.

Given an error log, I need to extract the failed status/reason for failure and the username, then concatenate them so the output can be sent in an email.

My output should look something like this:

Operation status: failed,Job Description not updated because this is not a matching Job Description ID.,username=FOO
Operation status: failed,Job Description not updated because this is not a matching Job Description ID.,username=BAR

Each line in the error log file closely resembles this:

"{ Operation status: failed,Job Description not updated because this is not a matching Job Description ID.,Sent Data:{lastAppraisalScore=0.0, country=null, jobTitle=LABORER..., username=FOO},...sendAcctActNotif=N}}"

There are no ellipses in the file; each line contains considerably more than is shown, but I have shown only the important parts.

Answer 1

A simple sed substitution would seem more adequate.

sed -n '/^"{ Operation status: failed,/!b
    s///;s/,Sent.*, username=/\t/;s/}.*//p' file

This searches for the first expression; if it is not found, we bypass this line. Otherwise, we substitute the matched string with nothing, and proceed to substitute away the other strings we don't want to keep, and finally print what remains.

Answer 2

Using this answer you can extract some part of your line (if it always looks the same way). Then you can process your file line by line :

while read line; do
  status=`grep -oP '(?<=Operation status:).*?(?=Sent)' <<< "$line"`
  user=`grep -oP '(?<=username=).*?(?=})' <<< "$line" | head -n 1`
  echo "Operation status:" $status "username="$user
done < file.txt

My example file.txt :

{ Operation status: failed1,Job Description not updated because this is not a matching Job Description ID.,Sent Data:{lastAppraisalScore=0.0, country=null, jobTitle=LABORER..., username=FOO1}, {username=BAR1}...sendAcctActNotif=N}}
{ Operation status: failed2,Job Description not updated because this is not a matching Job Description ID.,Sent Data:{lastAppraisalScore=0.0, country=null, jobTitle=LABORER..., username=FOO2}, {username=BAR2}...sendAcctActNotif=N}}
{ Operation status: failed3,Job Description not updated because this is not a matching Job Description ID.,Sent Data:{lastAppraisalScore=0.0, country=null, jobTitle=LABORER..., username=FOO3}, {username=BAR3}...sendAcctActNotif=N}}

My output :

Operation status: failed1,Job Description not updated because this is not a matching Job Description ID., username=FOO1
Operation status: failed2,Job Description not updated because this is not a matching Job Description ID., username=FOO2
Operation status: failed3,Job Description not updated because this is not a matching Job Description ID., username=FOO3