bash中的单词列表

Question

I want to do Unix script, witch print a list of uniq words form file and print list of numbers of rows where this word appearing.

For eample file.txt

Lorem 
ipsum dolor elit,
Lorem elit.

Output

Lorem 1,3
ipsum 2
dolor 2
elit 2,3

My code:

cat file.txt | tr '[:space:]' '[\n*]'| tr '[:digit:]' '[\n*]'| tr '[:punct:]' '[\n*]' | grep -v "^\s*$" | sort -f | uniq 

I don't know how I can do it... Someone can help me?

Answer 1

This awk codes works for your example:

awk '{for(i=1;i<=NF;i++){
        gsub(/[.,:;]/,"",$i)
        a[$i]=($i in a)?a[$i]","NR:NR}}
     END{for(x in a)print x,a[x]}' file

Answer 2

some write-only perl:

perl -nE '
    push @{$refs{$_}}, $. for /(\w+)/g
  } END { 
    say $_, "\t", join(",", @{$refs{$_}}) for keys %refs
' file

elit    2,3
Lorem   1,3
ipsum   2
dolor   2

It does not output the order of words as "when they when they were encountered in the file": the order is unspecified.

Also, if a word appears twice on one line, the line number will be added twice. To improve this:

perl -MList::Util=uniq -nE '
    push @{$refs{$_}}, $. for uniq /(\w+)/g
  } END { 
    say $_, "\t", join(",", @{$refs{$_}}) for keys %refs
' file

Answer 3

If you don't care that the words are in a different order than encountered in the file:

awk -F[^[:alpha:]] '{for (i=1; i<=NF;i++) 
                       if ($i) a[$i]=a[$i]?a[$i] "," NR:NR} 
                 END {for (e in a) print e,a[e]}' file

Or, if you want in the order encountered in the file:

awk -F[^[:alpha:]] 'FNR==NR{for (i=1; i<=NF;i++) 
                       if ($i) a[$i]=a[$i]?a[$i] "," NR:NR
                    next}
                    {for (i=1; i<=NF;i++){
                           if ($i in seen) continue 
                           else if ($i) {
                                   print $i,a[$i]
                                   seen[$i] } }
                     }' file file