使用“uniq -c”命令时需要从 output 中删除计数

Question

I am trying to read a file and sort it by number of occurrences of a particular field. Suppose i want to find out the most repeated date from a log file then i use uniq -c option and sort it in descending order. something like this

uniq -c | sort -nr 

This will produce some output like this -

809 23/Dec/2008:19:20

the first field which is actually the count is the problem for me.... i want to get ony the date from the above output but m not able to get this. I tried to use cut command and did this

uniq -c | sort -nr | cut -d' ' -f2 

but this just prints blank space... please can someone help me on getting the date only and chop off the count. I want only

23/Dec/2008:19:20

Thanks

Answer 1

The count from uniq is preceded by spaces unless there are more than 7 digits in the count, so you need to do something like:

uniq -c | sort -nr | cut -c 9-

to get columns (character positions) 9 upwards. Or you can use sed :

uniq -c | sort -nr | sed 's/^.\{8\}//'

or:

uniq -c | sort -nr | sed 's/^ *[0-9]* //'

This second option is robust in the face of a repeat count of 10,000,000 or more; if you think that might be a problem, it is probably better than the cut alternative. And there are undoubtedly other options available too.

---

Caveat: the counts were determined by experimentation on Mac OS X 10.7.3 but using GNU uniq from coreutils 8.3. The BSD uniq -c produced 3 leading spaces before a single digit count. The POSIX spec says the output from uniq -c shall be formatted as if with:

printf("%d %s", repeat_count, line);

which would not have any leading blanks. Given this possible variance in output formats, the sed script with the [0-9] regex is the most reliable way of dealing with the variability in observed and theoretical output from uniq -c :

uniq -c | sort -nr | sed 's/^ *[0-9]* //'

Answer 2

Instead of cut -d' ' -f2 , try

awk '{$1="";print}'

Maybe you need to remove one more blank in the beginning:

awk '{$1="";print}' | sed 's/^.//'

or completly with sed, preserving original whitspace:

sed -r 's/^[^0-9]*[0-9]+//'

Answer 3

Following awk may help you here.

awk '{a[$0]++} END{for(i in a){print a[i],i | "sort -k2"}}'  Input_file

Solution 2nd: In case you want order of output to be same as input but not as sort.

awk '!a[$0]++{b[++count]=$0} {c[$0]++} END{for(i=1;i<=count;i++){print c[b[i]],b[i]}}'  Input_file

Answer 4

an alternative solution is this:

uniq -c | sort -nr | awk '{print $1, $2}'

also you may easily print a single field.

Answer 5

use(since you use -f2 in the cut in your question)

cat file |sort |uniq -c | awk '{ print $2; }'

Answer 6

If you want to work with the count field downstream, following command will reformat it to a 'pipe friendly' tab delimited format without the left padding:

 .. | sort | uniq -c | sed -r 's/^ +([0-9]+) /\1\t/'

For the original task it is a bit of an overkill, but after reformatting, cut can be used to remove the field, as OP intended:

 .. | sort | uniq -c | sed -r 's/^ +([0-9]+) /\1\t/' | cut -d $'\t' -f2-

Answer 7

Add tr -s to the pipe chain to "squeeze" multiple spaces into one space delimiter:

uniq -c | tr -s ' ' | cut -d ' ' -f3

tr is very useful in some obscure places. Unfortunately it doesn't get rid of the first leading space, hence the -f3

Answer 8

You could make use of sed to strip both the leading spaces and the numbers printed by uniq -c

sort file | uniq -c | sed 's/^ *[0-9]* //'

I would illustrate this with an example. Consider a file

winebottles.mkv
winebottles.mov
winebottles.xges
winebottles.xges~
winebottles.mkv
winebottles.mov
winebottles.xges
winebottles.xges~

The command

sort file | uniq -c | sed 's/^ *[0-9]* //'

would return

winebottles.mkv
winebottles.mov
winebottles.xges
winebottles.xges~

Answer 9

first solution
just using sort when input repetition has not been taken into consideration. sort has unique option -u

• sort -u file
• sort -u < file

Ex.:

$ cat > file
a
b
c
a
a
g
d
d
$ sort -u file
a
b
c
d
g

second solution
if sort ing based on repetition is important

• sort txt | uniq -c | sort -k1 -nr | sed 's/^ \+[0-9]\+ //g'
• sort txt | uniq -c | sort -k1 -nr | perl -lpe 's/^ +[\d]+ +//g'

which has this output:

a
d
g
c
b