简体   繁体   English

Python 删除重复项并将它们写入新文件

[英]Python removing duplicates and writing them to a new file

I want to remove duplicate lines from a text file and write two new text files: 1 output file without duplicates, and another file that contains the lines which are duplicated in my original file.我想从文本文件中删除重复的行并编写两个新的文本文件:1 output 文件没有重复,另一个文件包含在我的原始文件中重复的行。

import re
import sys

lines_seen = set() # holds lines already seen
lines_seen.clear()
dups=open("dups.txt", "w")
outfile = open("out.txt", "w")
for line in open("input.txt", "r"):
    if line not in lines_seen: # not a duplicate
        outfile.write(line)
        lines_seen.add(line)

    else:

        dups.write(line)
lines_seen.clear()
outfile.close()
dups.close()

The output file is smaller than the original, which means that there are lines removed; output文件比原来的要小,说明有行被去掉了; however the duplicated file is empty, no duplicate lines are written.但是重复的文件是空的,没有重复的行被写入。

Because you're clearing the dups file and writing to it again, you need to append to it instead:因为您正在清除 dups 文件并再次写入它,所以您需要将 append 改为:

dups=open("dups.txt", "a")

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM