附加到具有字典理解的列表字典

Question

Suppose I have a large list of words. For an example:

>>> with open('/usr/share/dict/words') as f:
...     words=[word for word in f.read().split('\n') if word]

If I wanted to build an index by first letter of this word list, this is easy:

d={}
for word in words:
   if word[0].lower() in 'aeiou':
       d.setdefault(word[0].lower(),[]).append(word)
       # You could use defaultdict here too...

Results in something like this:

{'a':[list of 'a' words], 'e':[list of 'e' words], 'i': etc...}

Is there a way to do this with Python 2.7, 3+ dict comprehension? In other words, is it possible with the dict comprehension syntax to append the list represented by the key as the dict is being built?

ie:

  index={k[0].lower():XXX for k in words if k[0].lower() in 'aeiou'}

Where XXX performs an append operation or list creation for the key as index is being created.

Edit

Taking the suggestions and benchmarking:

def f1():   
    d={}
    for word in words:
        c=word[0].lower()
        if c in 'aeiou':
           d.setdefault(c,[]).append(word)

def f2():
   d={}
   {d.setdefault(word[0].lower(),[]).append(word) for word in words 
        if word[0].lower() in 'aeiou'} 

def f3():
    d=defaultdict(list)                       
    {d[word[0].lower()].append(word) for word in words 
            if word[0].lower() in 'aeiou'}         

def f4():
    d=functools.reduce(lambda d, w: d.setdefault(w[0], []).append(w[1]) or d,
       ((w[0].lower(), w) for w in words
        if w[0].lower() in 'aeiou'), {}) 

def f5():   
    d=defaultdict(list)
    for word in words:
        c=word[0].lower() 
        if c in 'aeiou':
            d[c].append(word)       

Produces this benchmark:

   rate/sec    f4     f2     f1     f3     f5
f4       11    -- -21.8% -31.1% -31.2% -41.2%
f2       14 27.8%     -- -11.9% -12.1% -24.8%
f1       16 45.1%  13.5%     --  -0.2% -14.7%
f3       16 45.4%  13.8%   0.2%     -- -14.5%
f5       18 70.0%  33.0%  17.2%  16.9%     --

The straight loop with a default dict is fastest followed by set comprehension and loop with setdefault .

Thanks for the ideas!

Answer 1

No - dict comprehensions are designed to generate non-overlapping keys with each iteration; they don't support aggregation. For this particular use case, a loop is the proper way to accomplish the task efficiently (in linear time).

Answer 2

It is not possible (at least easily or directly) with a dict comprehension.

It is possible, but potentially abusive of the syntax, with a set or list comprehension:

# your code:    
d={}
for word in words:
   if word[0].lower() in 'aeiou':
       d.setdefault(word[0].lower(),[]).append(word)        

# a side effect set comprehension:  
index={}   
r={index.setdefault(word[0].lower(),[]).append(word) for word in words 
        if word[0].lower() in 'aeiou'}     

print r
print [(k, len(d[k])) for k in sorted(d.keys())]  
print [(k, len(index[k])) for k in sorted(index.keys())]

Prints:

set([None])
[('a', 17094), ('e', 8734), ('i', 8797), ('o', 7847), ('u', 16385)]
[('a', 17094), ('e', 8734), ('i', 8797), ('o', 7847), ('u', 16385)]

The set comprehension produces a set with the results of the setdefault() method after iterating over the words list. The sum total of set([None]) in this case. It also produces your desired side effect of producing your dict of lists.

It is not as readable (IMHO) as the straight looping construct and should be avoided (IMHO). It is no shorter and probably not materially faster. This is more interesting trivia about Python than useful -- IMHO... Maybe to win a bet?

Answer 3

I'd use filter :

>>> words = ['abcd', 'abdef', 'eft', 'egg', 'uck', 'ice']
>>> index = {k.lower() : list(filter(lambda x:x[0].lower() == k.lower(),words)) for k in 'aeiou'}
>>> index
{'a': ['abcd', 'abdef'], 'i': ['ice'], 'e': ['eft', 'egg'], 'u': ['uck'], 'o': []}

Answer 4

This is not exactly a dict comprehension, but:

reduce(lambda d, w: d.setdefault(w[0], []).append(w[1]) or d,
       ((w[0].lower(), w) for w in words
        if w[0].lower() in 'aeiou'), {})

Answer 5

Not answering the question of a dict comprehension, but it might help someone searching this problem. In a reduced example, when filling growing lists on the run into a new dictionary, consider calling a function in a list comprehension, which is, admittedly, nothing better than a loop.

def fill_lists_per_dict_keys(k, v):
    d[k] = (
        v
        if k not in d 
        else d[k] + v
    )

# global d
d = {}
out = [fill_lists_per_dict_keys(i[0], [i[1]]) for i in d2.items()]

The out is only to suppress the None Output of each loop.

If you ever want to use the new dictionary even inside the list comprehension at runtime or if you run into another reason why your dictionary gets overwritten by each loop, check to make it global with global d at the beginning of the script (commented out because not necessary here).