Find count in the grep command in linux

Question

I have one log file of mongodb. I want to display all output of grep command in greater than or less than given value"protocol:op_msg 523ms"

sed -n '/2022-09-15T12:26/,/2022-09-15T14:03/p' mongod.log| grep "op_msg 523ms"

output of log file:

7391:2022-11-22T09:23:23.047-0500 I  COMMAND  [conn26] command test.test appName: "MongoDB Shell" command: find { find: "test", filter: { creationDate: new Date(1663252936409) }, lsid: { id: UUID("7c1bb40c-5e99-4281-9351-893e3d23261d") }, $clusterTime: { clusterTime: Timestamp(1669126970, 1), signature: { hash: BinData(0, B141BFD0978167F8C023DFB4AB32BBB117B3CD80), keyId: 7136078726260850692 } }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:337738 cursorExhausted:1 numYields:2640 nreturned:1 queryHash:6F9DC23E planCacheKey:6F9DC23E reslen:304 locks:{ ReplicationStateTransition: { acquireCount: { w: 2641 } }, Global: { acquireCount: { r: 2641 } }, Database: { acquireCount: { r: 2641 } }, Collection: { acquireCount: { r: 2641 } }, Mutex: { acquireCount: { r: 1 } } } storage:{ data: { bytesRead: 28615999, timeReadingMicros: 288402 } } protocol:op_msg 523ms

I have tried below command, but this command is only giving exact value. I need to find all query of log file which is greater than 100ms.

sed -n '/2022-09-15T12:26/,/2022-09-15T14:03/p' mongod.log| grep "op_msg 523ms"

Answer 1

Option 1) You can use grep bash onliner like this to filter mongo queries in specific execution time range:

grep -P '\d+ms' /log/mongodb/mongos.log  | while read LINE; do querytime="$(echo "$LINE" | grep -oP '\d+ms' | grep -oP '\d+')"; if [ "$querytime" -gt 6000 ]&&[ "$querytime" -lt 7000 ]; then echo "$LINE"; fi; done

Explained:

1. Filter all log lines having Number+"ms" at the end.
2. Loop over the queries to extract execution time in querytime variable
3. Check if $querytime between 6000ms and 7000ms
4. If $queritime in the specified range print the current $LINE

Option 2) You can use the --slow MS, --fast MS options from mlogfilter from the mtools package where you can do something like:

   mlogfilter mongod.log --slow 7000 --fast 6000

Answer 2

If all of the lines of interest have protocol:op_msg as the penultimate column, this becomes pretty trivial:

awk '$(NF-1) ~ /protocol:op_msg/ && $NF > 100 { print $NF }'

Note that this completely ignores the units when doing the comparison and will also print lines in which the final column is 250h or 250ns , but that's easy enough to filter for. With a more accurate problem description, a more precise solution is certainly available.