使用 awk 分析日志文件

Question

Hello there I have the following cut of a log file below:

Mon, 22 Mar 2020 13:15:39 +0200|185.34.66.225|user_1| - |user logged in| -
Mon, 22 Mar 2020 13:15:39 +0200|185.34.66.225|user_1| - |user changed password| -
Mon, 22 Mar 2020 13:15:39 +0200|185.34.66.225|user_1| - |user logged off| -
Mon, 22 Mar 2020 13:15:42 +0200|185.34.66.225|user_2| - |user logged in| -
Mon, 22 Mar 2020 13:15:40 +0200|185.34.66.215|user_3| - |user logged in| -
Mon, 22 Mar 2020 13:15:49 +0200|185.34.66.215|user_3| - |user changed password| -
Mon, 22 Mar 2020 13:15:49 +0200|185.34.66.215|user_3| - |user logged off| -
Mon, 22 Mar 2020 13:15:59 +0200|185.34.66.205|user_4| - |user logged in| -
Mon, 22 Mar 2020 13:15:59 +0200|185.34.66.205|user_4| - |user logged in| -
Mon, 22 Mar 2020 13:15:59 +0200|185.34.66.205|user_4| - |user changed password| -
Mon, 22 Mar 2020 13:15:59 +0200|185.34.66.205|user_4| - |user logged off| -
Mon, 22 Mar 2020 13:17:50 +0200|185.34.66.205|user_5| - |user logged in| -
Mon, 22 Mar 2020 13:17:50 +0200|185.34.66.205|user_5| - |user changed password| -
Mon, 22 Mar 2020 13:17:50 +0200|185.34.66.205|user_5| - |user changed profile| -
Mon, 22 Mar 2020 13:17:50 +0200|185.34.66.205|user_5| - |user logged off| -
Mon, 22 Mar 2020 15:19:19 +0200|178.56.66.225|user_6| - |user logged in| -
Mon, 22 Mar 2020 15:19:19 +0200|178.56.66.225|user_6| - |user changed password| -
Mon, 22 Mar 2020 15:19:19 +0200|178.56.66.225|user_6| - |user logged off| -
Mon, 22 Mar 2020 13:20:42 +0200|185.34.67.225|user_7| - |user logged in| -

the main idea is to get a list of bots who log in, change password, log off in the exact same second & without doing any other action between those 3 actions: I was able to achieve what I want using the following command:

cat /path/to/file | awk '{split($0,a,"|"); print a[3],a[1],a[5]}' | awk '{ print $6,$1,$8,$9,$10 }' | grep -A 1 -B 1 "user changed password" | awk 'seen[$1]++ ==2' | grep "user logged off" | awk '{ print $2}'

Output:

user_1
user_4
user_6

however I would need experts help to shorten my code & make it work as fast as possible in a huge log files

any help would be appreciated

Answer 1

Do everything in one awk call.

awk -F'|' '
  BEGIN {
    a[0]="user logged in"
    a[1]="user changed password"
    a[2]="user logged off"
  }
 lastuser!= $3 || lasttime!=$1 || a[expected]!=$5 {
   lasttime=$1
   lastuser=$3
   expected=(a[0]==$5?1:0)
   next
 }
 expected++==2 {
   print $3
 }' path_to_file

Answer 2

For your scenario, I think this will work pretty well

awk -F\| '{ vtAll[$3";"$1]++; if($5 ~ /user logged in|user logged off|user changed password/) vt[$3";"$1]++; } END { for (i in vt) if(vt[i] == 3 && vtAll[i] == 3) print i }' inputFile

To share my logic:

1. I created two arrays, having time and user as index
2. In vtAll, I save how many actions that user has made on that exact time
3. In vt, I check if the action is login, logoff or change pass. If so, I increment it as well
4. After the whole file was read, I check if there were three actions on both arrays. If there is, it means the user logged in, changed pass and logged off at the same time, and nothing else was made by that user.