[英]Python removing duplicates and writing them to a new file
I want to remove duplicate lines from a text file and write two new text files: 1 output file without duplicates, and another file that contains the lines which are duplicated in my original file.我想从文本文件中删除重复的行并编写两个新的文本文件:1 output 文件没有重复,另一个文件包含在我的原始文件中重复的行。
import re
import sys
lines_seen = set() # holds lines already seen
lines_seen.clear()
dups=open("dups.txt", "w")
outfile = open("out.txt", "w")
for line in open("input.txt", "r"):
if line not in lines_seen: # not a duplicate
outfile.write(line)
lines_seen.add(line)
else:
dups.write(line)
lines_seen.clear()
outfile.close()
dups.close()
The output file is smaller than the original, which means that there are lines removed; output文件比原来的要小,说明有行被去掉了; however the duplicated file is empty, no duplicate lines are written.
但是重复的文件是空的,没有重复的行被写入。
Because you're clearing the dups file and writing to it again, you need to append to it instead:因为您正在清除 dups 文件并再次写入它,所以您需要将 append 改为:
dups=open("dups.txt", "a")
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.