与使用Python的txt文件中的列表相比，如何从csv文件中删除行？

Question

I have a list of 12.000 dictionary entries (the words only, without their definitions) stored in a .txt file.

I have a complete dictionary with 62.000 entries (the words with their definitions) stored in .csv file.

I need to compare the small list in the .txt file with the larger list in the .csv file and delete the rows containing the entries that doesn't appear on the smaller list. In other words, I want to purge this dictionary to only 12.000 entries.

The .txt file is ordered in separate lines like this, line by line:

word1

word2

word3

The .csv file is ordered like this:

ID (column 1) WORD (column 2) MEANING (column 3)

How do I accomplish this using Python?

Answer 1

The following will not scale well, but should work for the number of records indicated.

import csv

csv_in = csv.reader(open(path_to_file, 'r'))
csv_out = csv.writer(open(path_to_file2, 'w'))
use_words = open(path_to_file3, 'r').readlines()

lookup = dict([(word, None) for word in use_words])

for line in csv_in:
    if lookup.has_key(line[0]):
        csv_out.writerow(line)

csv_out.close()

Answer 2

Good answers so far. If you want to get minimalistic...

import csv

lookup = set(l.strip().lower() for l in open(path_to_file3))
map(csv.writer(open(path_to_file2, 'w')).writerow, 
    (row for row in csv.reader(open(path_to_file)) 
    if row[1].lower() in lookup))

Answer 3

One of the least known facts of current computers is that when you delete a line from a text file and save the file, most of the time the editor does this:

1. load the file into memory
2. write a temporary file with the rows you want
3. close the files and move the temp over the original

So you have to load your wordlist:

with open('wordlist.txt') as i:
    wordlist = set(word.strip() for word in i)  #  you said the file was small

Then you open the input file:

with open('input.csv') as i:
    with open('output.csv', 'w') as o:
        output = csv.writer(o)
        for line in csv.reader(i):  # iterate over the CSV line by line
            if line[1] not in wordlist:  # test the value at column 2, the word
                output.writerow(line) 

os.rename('input.csv', 'output.csv')

This is untested, now go do your homework and comment here if you find any bug... :-)

Answer 4

i would use pandas for this. the data set's not large, so you can do it in memory with no problem.

import pandas as pd

words = pd.read_csv('words.txt')
defs = pd.read_csv('defs.csv')
words.set_index(0, inplace=True)
defs.set_index('WORD', inplace=True)
new_defs = words.join(defs)
new_defs.to_csv('new_defs.csv')

you might need to manipulate new_defs to make it look like you want it to, but that's the gist of it.