为什么我收到空字典？

Question

I am learning python from an introductory Python textbook and I am stuck on the following problem:

You will implement function index() that takes as input the name of a text file and a list of words. For every word in the list, your function will find the lines in the text file where the word occurs and print the corresponding line numbers.

Ex:

 >>>> index('raven.txt', ['raven', 'mortal', 'dying', 'ghost', 'ghastly', 'evil', 'demon'])

 ghost     9 
 dying     9 
 demon     122
 evil      99, 106
 ghastly   82
 mortal    30 
 raven     44, 53, 55, 64, 78, 97, 104, 111, 118, 120

Here is my attempt at the problem:

def index(filename, lst):
    infile = open(filename, 'r')
    lines =  infile.readlines()
    lst = []
    dic = {}
    for line in lines:
        words = line.split()
        lst. append(words)
    for i in range(len(lst)):
        for j in range(len(lst[i])):
            if lst[i][j] in lst:
                dic[lst[i][j]] = i 
    return dic

When I run the function, I get back an empty dictionary. I do not understand why I am getting an empty dictionary. So what is wrong with my function? Thanks.

Answer 1

Try this,

def index(filename, lst):
    dic = {w:[] for w in lst}
    for n,line in enumerate( open(filename,'r') ):
        for word in lst:
            if word in line.split(' '):
                dic[word].append(n+1)
    return dic

There are some features of the language introduced here that you should be aware of because they will make life a lot easier in the long run.

The first is a dictionary comprehension. It basically initializes a dictionary using the words in lst as keys and an empty list [] as the value for each key.

Next the enumerate command. This allows us to iterate over the items in a sequence but also gives us the index of those items. In this case, because we passed a file object to enumerate it will loop over the lines. For each iteration, n will be the 0-based index of the line and line will be the line itself. Next we iterate over the words in lst .

Notice that we don't need any indices here. Python encourages looping over objects in sequences rather than looping over indices and then accessing the objects in a sequence based on index (for example discourages doing for i in range(len(lst)): do something with lst[i]) .

Finally, the in operator is a very straightforward way to test membership for many types of objects and the syntax is very intuitive. In this case, we are asking is the current word from lst in the current line .

Note that we use line.split(' ') to get a list of the words in the line. If we don't do this, 'the' in 'there was a ghost' would return True as the is a substring of one of the words.

On the other hand 'the' in ['there', 'was', 'a', 'ghost'] would return False . If the conditional returns True , we append it to the list associated to the key in our dictionary.

That might be a lot to chew on, but these concepts make problems like this more straight forward.

Answer 2

You are overwriting the value of lst . You use it as both a parameter to a function (in which case it is a list of strings) and as the list of words in the file (in which case it's a list of list of strings). When you do:

if lst[i][j] in lst

The comparison always returns False because lst[i][j] is a str , but lst contains only lists of strings, not strings themselves. This means that the assignment to the dic is never executed and you get an empty dict as result.

To avoid this you should use a different name for the list in which you store the words, for example:

In [4]: !echo 'a b c\nd e f' > test.txt

In [5]: def index(filename, lst):
   ...:     infile = open(filename, 'r')
   ...:     lines =  infile.readlines()
   ...:     words = []
   ...:     dic = {}
   ...:     for line in lines:
   ...:         line_words = line.split()
   ...:         words.append(line_words)
   ...:     for i in range(len(words)):
   ...:         for j in range(len(words[i])):
   ...:             if words[i][j] in lst:
   ...:                 dic[words[i][j]] = i 
   ...:     return dic
   ...: 

In [6]: index('test.txt', ['a', 'b', 'c'])
Out[6]: {'a': 0, 'c': 0, 'b': 0}

---

There are also a lot of things you can change.

When you want to iterate a list you don't have to explicitly use indexes. If you need the index you can use enumerate :

    for i, line_words in enumerate(words):
        for word in line_words:
            if word in lst: dict[word] = i

You can also iterate directly on a file (refer to Reading and Writing Files section of the python tutorial for a bit more information):

# use the with statement to make sure that the file gets closed
with open('test.txt') as infile:
    for i, line in enumerate(infile):
        print('Line {}: {}'.format(i, line))

In fact I don't see why would you first build that words list of list. Just itertate on the file directly while building the dictionary:

def index(filename, lst):
    with open(filename, 'r') as infile:
        dic = {}
        for i, line in enumerate(infile):
            for word in line.split():
                if word in lst:
                    dic[word] = i 
    return dic

---

Your dic values should be lists, since more than one line can contain the same word. As it stands your dic would only store the last line where a word is found:

from collections import defaultdict

def index(filename, words):
    # make faster the in check afterwards
    words = frozenset(words)  
    with open(filename) as infile:
        dic = defaultdict(list)
        for i, line in enumerate(infile):
            for word in line.split():
                if word in words:
                    dic[word].append(i)
    return dic

If you don't want to use the collections.defaultdict you can replace dic = defaultdict(list) with dic = {} and then change the:

dic[word].append(i)

With:

if word in dic:
    dic[word] = [i]
else:
    dic[word].append(i)

Or, alternatively, you can use dict.setdefault :

dic.setdefault(word, []).append(i)

although this last way is a bit slower than the original code.

Note that all these solutions have the property that if a word isn't found in the file it will not appear in the result at all. However you may want it in the result, with an emty list as value. In such a case it's simpler the dict with empty lists before starting to loop, such as in:

dic = {word : [] for word in words}
for i, line in enumerate(infile):
    for word in line.split():
        if word in words:
            dic[word].append(i)

Refer to the documentation about List Comprehensions and Dictionaries to understand the first line.

You can also iterate over words instead of the line, like this:

dic = {word : [] for word in words}
for i, line in enumerate(infile):
    for word in words:
        if word in line.split():
            dic[word].append(i)

Note however that this is going to be slower because:

• line.split() returns a list, so word in line.split() will have to scan all the list.
• You are repeating the computation of line.split() .

You can try to solve these two problems doing:

dic = {word : [] for word in words}
for i, line in enumerate(infile):
    line_words = frozenset(line.split())
    for word in words:
        if word in line_words:
            dic[word].append(i)

Note that here we are iterating once over line.split() to build the set and also over words . Depending on the sizes of the two sets this may be slower or faster than the original version (iteratinv over line.split() ).

However at this point it's probably faster to intersect the sets:

dic = {word : [] for word in words}
for i, line in enumerate(infile):
    line_words = frozenset(line.split())
    for word in words & line_words:  # & stands for set intersection
        dic[word].append(i)

Answer 3

First, your function param with the words is named lst and also the list where you put all the words in the file is also named lst , so you are not saving the words passed to your functions, because on line 4 you're redeclaring the list.

Second, You are iterating over each line in the file (the first for ), and getting the words in that line. After that lst has all the words in the entire file. So in the for i ... you are iterating over all the words readed from the file, there's no need to use the third for j where you are iterating over each character in every word.

In resume, in that if you are saying " If this single character is in the lists of words ... " wich is not, so the dict will be never filled up.

for i in range(len(lst)):
  if words[i] in lst:
    dic[words[i]] = dic[words[i]] + i  # To count repetitions

You need to rethink the problem, even my answer will fail because the word in the dict will not exist giving an error, but you get the point. Good luck!