简体   繁体   中英

append distinct count for each line in a file in unix

I have a big file with field separator #~# and records like

bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#103  
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#115
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#117
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#103 -> repeat of 1st
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#118
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#129
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#130
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#132
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#133

How can I process this file to record the occurrence count as below

bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#103#~#2 
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#115#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#117#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#118#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#129#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#130#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#132#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#133#~#1

Thank You.

You can use below:

sort filename |uniq -c|awk '{print $2,$3"#~#"$1}'

Output:

bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#103#~#2
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#115#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#117#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#118#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#129#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#130#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#132#~#1
bus#~#337007270#~#461692988#~#2019-09-24 01:06:33#~#133#~#1

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM