在整个词典中错误地重复了子词典？

Question

I'm trying to store in a dictionary the number of times a given letter occurs after another given letter. For example, dictionary['a']['d'] would give me the number of times 'd' follows 'a' in short_list .

alphabet = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']
short_list = ['ford','hello','orange','apple']

# dictionary to keep track of how often a given letter occurs
tally = {}
for a in alphabet:
    tally[a] = 0

# dictionary to keep track of how often a given letter occurs after a given letter 
# e.g. how many times does 'd' follow 'a' -- master_dict['a']['d']
master_dict = {}
for a in alphabet:
    master_dict[a] = tally

def precedingLetter(letter,word):
    if word.index(letter) == 0:
         return
    else:
         return word[word.index(letter)-1]

for a in alphabet:
    for word in short_list:
        for b in alphabet:
            if precedingLetter(b,word) == a:
                 master_dict[a][b] += 1

However, the entries for all of the letters (the keys) in master_dict are all the same. I can't think of another way to properly tally each letter's occurrence after another letter. Can anyone offer some insight here?

Answer 1

If the sub- dict s are all supposed to be updated independently after creation, you need to shallow copy them. Easiest/fastest way is with .copy() :

for a in alphabet:
    master_dict[a] = tally.copy()

The other approach is to initialize the dict lazily. The easiest way to do that is with defaultdict :

from collections import defaultdict

masterdict = defaultdict(lambda: defaultdict(int))

# or

from collections import Counter, defaultdict

masterdict = defaultdict(Counter)

No need to pre-create empty tallies or populate masterdict at all, and this avoids creating dict s when the letter never occurs. If you access masterdict[a] for an a that doesn't yet exist, it creates a defaultdict(int) value for it automatically. When masterdict[a][b] is accessed and doesn't exist, the count is initialized to 0 automatically.

Answer 2

In Addition to the first answer it could be handy to perform your search the other way around. So instead of looking for each possible pair of letters, you could iterate just over the words.

In combination with the defaultdict this could simplify the process. As an example:

from collections import defaultdict

short_list = ['ford','hello','orange','apple']
master_dict = defaultdict(lambda: defaultdict(int))

for word in short_list:
    for i in range(0,len(word)-1):
        master_dict[word[i]][word[i+1]] += 1

Now master_dict contains all occured letter combinations while it returns zero for all other ones. A few examples below:

print(master_dict["f"]["o"]) # ==> 1
print(master_dict["o"]["r"]) # ==> 2
print(master_dict["a"]["a"]) # ==> 0

Answer 3

The problem you ask about is that the master_dict[a] = tally is only assigning the same object another name, so updating it through any of the references updates them all. You could fix that by making a copy of it each time by using master_dict[a] = tally.copy() as already pointed out in @ShadowRanger's answer .

As @ShadowRanger goes on to point out, it would also be considerably less wasteful to make your master_dict a defaultdict(lambda: defaultdict(int)) because doing so would only allocate and initialize counts for the combinations that actually encountered rather than all possible 2 letter permutations (if it was used properly).

To give you a concert idea of the savings, consider that there are only 15 unique letter pairs in your sample short_list of words, yet the exhaustive approach would still create and initialize 26 placeholders in 26 dictionaries for all 676 the possible counts.

It also occurs to me that you really don't need a two-level dictionary at all to accomplish what you want since the same thing could be done with a single dictionary which had keys comprised of tuples of pairs of characters.

Beyond that, another important improvement, as pointed out in @AdmPicard's answer , is that your approach of iterating through all possible permutations and seeing if any pairs of them are in each word via the precedingLetter() function is significantly more time consuming than it would be if you just iterated over all the successive pairs of letters that actually occurred in each one of them.

So, putting all this advice together would result in something like the following:

from collections import defaultdict
from string import ascii_lowercase

alphabet = set(ascii_lowercase)
short_list = ['ford','hello','orange','apple']
# dictionary to keep track of how often a letter pair occurred after one other. 
# e.g. how many times 'd' followed an 'a' -> master_dict[('a','d')]
master_dict = defaultdict(int)

try:
    from itertools import izip
except ImportError:  # Python 3
    izip = zip

def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = iter(iterable), iter(iterable)  # 2 independent iterators
    next(b, None)                          # advance the 2nd one
    return izip(a, b)

for word in short_list:
    for (ch1,ch2) in pairwise(word.lower()):
        if ch1 in alphabet and ch2 in alphabet:
            master_dict[(ch1,ch2)] += 1

# display results
unique_pairs = 0
for (ch1,ch2) in sorted(master_dict):
    print('({},{}): {}'.format(ch1, ch2, master_dict[(ch1,ch2)]))
    unique_pairs += 1

print('A total of {} different letter pairs occurred in'.format(unique_pairs))
print('the words: {}'.format(', '.join(repr(word) for word in short_list)))

Which produces this output from the short_list :

(a,n): 1
(a,p): 1
(e,l): 1
(f,o): 1
(g,e): 1
(h,e): 1
(l,e): 1
(l,l): 1
(l,o): 1
(n,g): 1
(o,r): 2
(p,l): 1
(p,p): 1
(r,a): 1
(r,d): 1

A total of 15 different letter pairs occurred in
the words: 'ford', 'hello', 'orange', 'apple'