从文本文件中删除两个重复项（而不仅仅是重复项）？

Question

By this I mean, erase all rows in a text file that are repeated, NOT just the duplicates. I mean both the row that is duplicated and the duplicated row. This would leave me only with the list of rows that weren't repeated. Perhaps a regular expression could do this in notepad++? But which one? Any other methods?

Answer 1

If you're on a unix-like system, you can use the uniq command.

ezra@ubuntu:~$ cat test.file
ezra
ezra
john
user
ezra@ubuntu:~$ uniq -u test.file
john
user

Note, that the similar rows be adjacent. You'll have to sort the file first if they're not.

ezra@ubuntu:~$ cat test.file
ezra
john
ezra
user
ezra@ubuntu:~$ uniq -u test.file
ezra
john
ezra
user
ezra@ubuntu:~$ sort test.file | uniq -u
john
user

Answer 2

If you have acess to a regex that supports PCRE style, this is straight forward:

s/(?:^|(?<=\\n))(.*)\\n(?:\\1(?:\\n|$))+//g

(?:^|(?<=\n))     # Behind us is beginning of string or newline
(.*)\n            # Capture group 1: all characters up until next newline
(?:               # Start non-capture group
    \1                # backreference to what was captured in group 1
    (?:\n|$)          # a newline or end of string
)+                # End non-capture group, do this 1 or more times

Context is a single string

use strict; use warnings;

my $str = 
'hello
this is
this is
this is
that is';

$str =~ s/
          (?:^|(?<=\n))
          (.*)\n
          (?:
              \1
              (?:\n|$)
          )+
  //xg;

print "'$str'\n";

__END__

output:

'hello
that is'