bash 脚本在具有索引的文件中查找唯一值

Question

I have a text file,test.txt, like

shekhar cbv
ravi cbv
ravi sdf
asd df
ravi Df
ravi dfg
ravi df
ravi dfg
ravi df
afas cvb
sdf hgh
sasdg cfg

I want to sort based on 2nd field and want to write 2nd field only on "result.txt" Currently I am doing following:

sort -k 2,2 test.txt| uniq -i -f 1 | cut -d ' ' -f2 > result.txt

which is writing 2n field ie

cbv 
cfg 
cvb 
df  
dfg 
hgh 
sdf 

Corresponding to every unique 2nd field value I want its,all, index in original file. How this is possible?

Desired Output:

cbv 1 2
cfg 12
cvb 10
df 4 5 7 9 
dfg 6 8
hgh 11
sdf 3

One more thing if say there is 3rd field also. How to achieve the above only mean while sorting and finding uniqueness only 2nd field is used.

input With 3rd field

    shekhar cbv rg
    ravi cbv fdf
    ravi sdf dfh
    asd df dfhdfh
    ravi Df fgh
    ravi dfg dfh
    ravi df dfgh
    ravi dfg dfgh
    ravi df dfhg
    afas cvb fhfg
    sdf hgh cgfhfg
    sasdg cfg fgh

Desired o/p same. Thanks, Ravi

Answer 1

Try this command to print column with all original indexes :

awk '{k=tolower($2); arr[k]=arr[k] " " NR} END{for(v in arr) print v, arr[v]}' test.txt | sort -f -k 1,1

OUTPUT

cbv  1 2
cfg  12
cvb  10
df  4 5 7 9
dfg  6 8
hgh  11
sdf  3

Update: using awk only solution

awk '{k=tolower($2); arr[k]=arr[k] " " NR} END{n=asorti(arr, dest); for(i = 1; i <= n; i++) print dest[i], arr[dest[i]]}' test.txt

Your file can have any number of columns but this command will only look at 2nd column.

Answer 2

I think you want to use cut to extract the column you want and then do the sort and uniq stuff:

cut -f2 -d' ' test.txt | sort -f | uniq -i > result.txt

This assumes that the columns are separated by a single space.

Note that you'll want -f switch on sort so that the sorting will be case insensitive, otherwise rows that differ only in case won't be beside each other and uniq -i probably won't do what you want it to do.

Answer 3

You're so close, To get the value of the second column based on what you've done so far, you should use awk . It is made for processing a stream line by line and extracting just the parts you want.

Your code: sort -k 2,2 test.txt| uniq -i -f 1 | cut -d ' ' -f2 > result.txt sort -k 2,2 test.txt| uniq -i -f 1 | cut -d ' ' -f2 > result.txt

With awk: sort -k 2,2 test.txt| uniq -i -f 1 | cut -d ' ' -f2 | awk '{print $2}' > result.txt sort -k 2,2 test.txt| uniq -i -f 1 | cut -d ' ' -f2 | awk '{print $2}' > result.txt

Awk will split your input by whitespace, and print $2 will take the second text block. I'd recommend looking into awk - it is great for many problems.

Answer 4

For fun - perl:

perl -anle 'push(@{$s{$F[1]}},++$n);END{map{print "$_: @{$s{$_}}"} sort keys %s}'

or case insensitive

perl -anle 'push(@{$s{lc($F[1])}},++$n);END{map{print "$_: @{$s{$_}}"} sort keys %s}'