unix shell脚本-搜索字符串并格式化为json

Question

I have a log file with multiple json message that is not formatted. Trying to write a shell script to search for a string in the logs and format the matching json and write to a file.

For Example :- in the test.log file,

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService - {"header":{"time":"2015 06:06:13 PM EDT","reqId":"1","user":"baskar"}}

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService - {"header":{"time":"2015 06:06:13 PM EDT","reqId":"2","user":"raja"}}

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService - {"header":{"time":"2015 06:06:13 PM EDT","reqId":"3","user":"baskar"}}

I use the below command,

grep -i "baskar" test.log | grep -o "{\"header\".*" | python -m json.tool > ~/search_result.log

The above command works fine if the search returns only one json message. If there are multiple matches, this doesnt work and throw the following error.

Unable to decode the json message.

Whereas, if i redirect the matched output to some file and then read the file line by line to format the json works.

 grep -i "baskar" test.log | grep -o "{\"header\".*" > ~/search_result.log

while read line
do
    name=$line    
    echo $name | python -m json.tool >> ~/formatted_search_result.log
done < ~/search_result.log

Since, writing to a temp file and then reading from the temp file to format the json hits the performance, looking for some efficient way of doing this.

The expected output is, When i search for

grep -i "baskar" test.log

,

{
    "header": {
      "user": "baskar",
      "reqId": "1",
      "time": "2015 06:06:13 PM EDT"
    }
  }
  {
    "header": {
      "user": "baskar",
      "reqId": "3",
      "time": "2015 06:06:13 PM EDT"
    }

}

Thanks, Baskar.S

Answer 1

Use jq . Given your sample text:

sed 's/.*Service - //' file | jq -s .

Remove the -s option if you don't want an array of objects.

---

You can filter the input to jq:

sed -n '/baskar/ s/.*Service - //p' file | jq  .

or let jq to the filtering

sed 's/.*Service - //' file | jq  'select(.header.user == "baskar")'

Either option produces

{
  "header": {
    "user": "baskar",
    "reqId": "1",
    "time": "2015 06:06:13 PM EDT"
  }
}
{
  "header": {
    "user": "baskar",
    "reqId": "3",
    "time": "2015 06:06:13 PM EDT"
  }
}