我如何解析Linux中的大文件

Question

我是Linux的初學者。 我有以下平面文件test.txt

Iteration 1   
Telephony   


Pass/Fail

5.1.1.1   voiceCallPhoneBook   50   45
5.1.1.4   voiceCallPhoneHistory   50   49
5.1.1.7   receiveCall   100   100
5.1.1.8   deleteContacts   20   19
5.1.1.9   addContacts   20   20
Telephony   16:47:42
Messaging   


Pass/Fail

5.1.2.3   openSMS   50   49
5.1.2.1   smsManuallyEntryOption   50   50
5.1.2.2   smsSelectContactsOption   50   50
Messaging   03:26:31
Email   


Pass/Fail

Email   00:00:48
Email   


Pass/Fail

Email   00:00:40
PIM   


Pass/Fail

5.1.6.1   addAppointment   5   0
5.1.6.2   setAlarm   1   0
5.1.6.3   deleteAppointment   5   0
5.1.6.4   deleteAlarm   1   0
5.1.6.5   addTask   1   0
5.1.6.6   openTask   1   0
5.1.6.7   deleteTask   1   0
PIM   00:03:06
Multi-Media   

teration 2   
Telephony   


Pass/Fail

5.1.1.1   voiceCallPhoneBook   50   47
5.1.1.4   voiceCallPhoneHistory   50   50
5.1.1.7   receiveCall   100   100
5.1.1.8   deleteContacts   20   20
5.1.1.9   addContacts   20   20
Telephony   04:02:05
Messaging   


Pass/Fail

5.1.2.3   openSMS   50   50
5.1.2.1   smsManuallyEntryOption   50   50
5.1.2.2   smsSelectContactsOption   50   50
Messaging   03:20:01
Email   


Pass/Fail

Email   00:00:47
Email   


Pass/Fail

Email   00:00:40
PIM   


Pass/Fail

5.1.6.1   addAppointment   5   5
5.1.6.2   setAlarm   1   1
5.1.6.3   deleteAppointment   5   5
5.1.6.4   deleteAlarm   1   1
5.1.6.5   addTask   1   1
5.1.6.6   openTask   1   1
5.1.6.7   deleteTask   1   1
PIM   00:09:20
Multi-Media   

我想計算文件中特定單詞出現的次數，例如：如果我搜索“ voiceCallPhoneBook”，它將顯示為2次。

我可以用

cat reports.txt | grep "5.1.1.4" | cut -d' ' -f1,4,7,10 |

運行此腳本后，我得到如下輸出

5.1.1.4 voiceCallPhoneBook  50  45
5.1.1.4 voiceCallPhoneBook  50  47

這是一個非常大的文件，我想使用帶有bash / awk腳本的循環，還可以找到第三列和第四列值之和的平均值。 我正在努力用bash腳本編寫。 希望有人可以為此提供解決方案。

謝謝

Answer 1

#!/usr/bin/awk -f
BEGIN{
c3 = 0
c4 = 0
count = 0
}

/voiceCallPhoneBook/{
    c3 = c3 + $3;
    c4 = c4 + $4;
    count++;
}

END{
 print "column 3 avg: " c3/count
 print "column 4 avg: " c4/count
}

1）將其保存在文件中，例如countVoiceCall.awk

2）awk -f countVoiceCall.awk sample.txt

輸出：

column 3 avg: 50
column 4 avg: 46

簡要說明：

a.    BEGIN{...} block uses for variables initialization
b.    /PATTERN/{...}  blocks uses to search your keyword, for example "voiceCallPhoneBook"
c.    END{...} block uses for print the results

Answer 2

這將搜索包含5.1.1.4的行
匯總第三列和第四列
然后將它們全部打印出來

awk '/^5\.?\.?\.?/ {a[$1" " $2] +=$3 ; b[$1" " $2] +=$4 }
END{ for (k in a){
      printf("%-50s%-10i%-10i\n",k,a[k],b[k])}
}' $1

與今天早些時候重復的內容在這里使用awk解析大型測試文件

帶有標題avg和出現次數，並格式化得更整潔，以便於閱讀：)

awk 'BEGIN{
            printf("%-50s%-10s%-10s%-10s\n","Name","Col3 Tot","Col4 Tot","Ocurr")
}

/^5\.?\.?\.?/ {
        count++
        c3 = c3 + $3
        c4 = c4 + $4
        a[$1" " $2] +=$3
        b[$1" " $2] +=$4
        c[$1" " $2]++
}

END{
        for (k in a)
                {printf("%-50s%-10i%-10i%-10i\n",k,a[k],b[k],c[k])}

                print "col3 avg: " c3/count "\ncol4 avg: " c4/count
}' $1