在一个文件中使用文本在第二个文件中搜索匹配项

Question

I'm using python 2.6 on linux.

I have two text files first.txt has a single string of text on each line. So it looks like

lorem
ipus
asfd

The second file doesn't quite have the same format. it would look more like this

1231 lorem
1311 assss 31 1
etc

I want to take each line of text from first.txt and determine if there's a match in the second text. If there isn't a match then I would like to save the missing text to a third file. I would like to ignore case but not completely necessary. This is why I was looking at regex but didn't have much luck.

So I'm opening the files, using readlines() to create a list.
Iterating through the lists and printing out the matches.

Here's my code

first_file=open('first.txt', "r")
first=first_file.readlines()
first_file.close()

second_file=open('second.txt',"r")
second=second_file.readlines()
second_file.close()

while i < len(first):
  j=search[i]
  while k < len(second):
   m=compare[k]
   if not j.find(m):
    print m
   i=i+1
   k=k+1
exit() 

It's definitely not elegant. Anyone have suggestions how to fix this or a better solution?

Answer 1

My approach is this: Read the second file, convert it into lowercase and then create a list of the words it contains. Then convert this list into a set , for better performance with large files.

Then go through each line in the first file, and if it (also converted to lowercase, and with extra whitespace removed) is not in the set we created, write it to the third file.

with open("second.txt") as second_file:
    second_values = set(second_file.read().lower().split())

with open("first.txt") as first_file:
    with open("third.txt", "wt") as third_file:
        for line in first_file:
            if line.lower().strip() not in second_values:
                third_file.write(line + "\n")

set objects are a simple container type that is unordered and cannot contain duplicate value. It is designed to allow you to quickly add or remove items, or tell if an item is already in the set.

with statements are a convenient way to ensure that a file is closed, even if an exception occurs. They are enabled by default from Python 2.6 onwards, in Python 2.5 they require that you put the line from __future__ import with_statements at the top of your file.

The in operator does what it sounds like: tell you if a value can be found in a collection. When used with a list it just iterates through, like your code does, but when used with a set object it uses hashes to perform much faster. not in does the opposite. (Possible point of confusion: in is also used when defining a for loop ( for x in [1, 2, 3] ), but this is unrelated.)

Answer 2

Assuming that you're looking for the entire line in the second file:

second_file=open('second.txt',"r")
second=second_file.readlines()
second_file.close()


first_file=open('first.txt', "r")
for line in first_file:
    if line not in second:
        print line

first_file.close()