多级字典python

Question

I want to make a multi-level dictionary, which is the count of times the token has appeared with a Pos tag.

example:

cat/nt     
cat = token. nt = pos tag 

I have gotten up to here but I'm stuck.

import re 

dicts = {}

wds = re.compile('(\w*|\w+\.\w*)([/])(\w+)')

with open('train.txt', 'r') as td:

for lines in td:

m =  wds.finditer(lines)

for mms in m:

dicts[mms.group(1)] = mms.group(3)

content of train.txt file

Pierre/NNP Vinken/NNP ,/, 61/CD years/NNS old/JJ ,/, will/MD join/VB

Answer 1

Try something like this in your inner for loop:

for mms in m:
    token = mms.group(1)
    pos = mms.group(3)
    if token in dicts:
        if pos in dicts[token]:
            dicts[token][pos] += 1
        else:
            dicts[token][pos] = 1
    else:
        dicts[token] = {pos: 1}

This checks if we've seen the token at all before, and given that we have, checks if we've seen it as this part of speech before. If we've seen this combination before, increment its count. If we've seen the token, but not this POS, default it to a count of 1. If we've never even seen the token before, add an entry for it with a sub-dict containing this POS at a count of 1.

You could get the same effect with a default dict, but I thought seeing how it worked behind the scenes would be clearer.

EDIT: To print the resulting dict, try

for token in dicts.keys():
    for pos in dicts[token].keys():
        print "%s %s: %s" % (token, pos, dicts[token][pos])