解析字典时的python KeyError

Question

How can i join these two text documents?

document 1:

1000001 10:0.471669 250:0.127552 30:0.218773 64:0.249413
1000002 130:0.0839656 107:0.185613 30:0.446355 110:0.38011
1000003 1:0.0835855 1117:0.0647112 302:0.0851354 46:0.0601825 48:0.098907 516:0.167713

document 2:

1000001 161:0.115664 207:0.136537 294:0.0974809 301:0.199868
1000002
1000003 555:0.0585849 91:0.0164101

result:

1000001 10:0.471669 250:0.127552 30:0.218773 64:0.249413 161:0.115664 207:0.136537 294:0.0974809 301:0.199868
1000002 130:0.0839656 107:0.185613 30:0.446355 110:0.38011
1000003 1:0.0835855 1117:0.0647112 302:0.0851354 46:0.0601825 48:0.098907 516:0.167713 555:0.0585849 91:0.0164101

explanation:
document 1 and document 2 both have the same structure and they have the same number of lines.
Each line starts with a number (the same number in both documents), and then we have several items in each line which are made up of a number+colon+a decimal number:
example 10:0.471669
these item combinations are unique, what I want to do is to merge them together: take the items from the second document for each line and put it in the corresponding line of the first document.
note:
the initial number at the beginning and the items from one another are separated by a single space.

update

here is my try:

dat1 = {}
with open('doc1') as f:
    for line in f.readlines():
        dat1[line.split(' ')[0]] = line.strip().split(' ')[1:]

dat2 = {}
with open('doc2') as f:
    for line in f.readlines():
        key = line.split(' ')[0]
        dat2[key] = line.split(' ')[1:]

for key in dat1.keys():
    print("%s %s %s" % (key, str.join(' ', dat1[key]), str.join(' ', dat2[key])))

i get a traceback of KeyError, on the lines of the second document when the line doens't have anything to be added to the first document. It is the case in the second line of the second document in the above example.
How can I escape this exception? escape the lines which have only the key and nothing else to add?

Answer 1

An easier way might be to use a defaultdict of lists:

from collections import defaultdict

data = defaultdict(list)

for filename in 'doc1', 'doc2':
    with open(filename) as f:
        for line in f:
            key, _, value = line.partition(' ')
            data[key.strip()].append(value.strip())

for key in sorted(data):
    print key, ' '.join(data[key])    # Python 2
#    print(key, *data[key])            # Python 3

Regarding the printing of the result you could add:

from __future__ import print_function

to the top of your file, and then the Python 3 print() function will be available in Python 2, ie you can use the Python 3 print above.

---

You asked in a comment how to print to a file (Python 3, or Python 2 after importing print_function ):

with open('outfile.txt', 'w') as f:
    for key in sorted(data):
        print(key, *data[key], file=f)

Answer 2

The problem is with newline characters.

At the end of each line in the file there is a newline character which will be included in the last entry of each line. The exception occurs because dat1 will have a key "1000002" and dat2 will have a key "1000002\\n" .

If you have line = line.strip() before parsing then the code should work as expected.

for line in f.readlines():
    line = line.strip()
    key = line.split(' ')[0]
    dat2[key] = line.split(' ')[1:]

Answer 3

You can use pop operation to get the first item of an array, like this:

def read_stem(f):
        res = {}
        for line in f.readlines():
                items = line.strip().split()
                res[items.pop(0)] = items
        return res

with open('stem.data') as f:
        dat1 = read_stem(f)

with open('stem.info') as f:
        dat2 = read_stem(f)

with open('myfile','w') as f:
    for key in dat1.keys():
            f.write("%s %s\n" % (key, ' '.join(dat1[key] + dat2[key])))

Answer 4

In your code in 2nd file key for empty row was '1000002\\n' not 1000002, that could be the reason, this works.

file1_lines= open('doc1', 'r').readlines()
file2_lines = open('doc1', 'r').readlines()
resfile = open('res.txt','w')


dat1 = {}
for line in file1_lines:
    dat1[line.split(' ')[0]] = line.strip().split(' ')[1:]

dat2 = {}
for line in file2_lines:
    dat2[line.strip().split(' ')[0]] = line.strip().split(' ')[1:]

print(dat1)
print(dat2)

for key in dat1.keys():
    print("%s %s %s" % (key, str.join(' ', dat1[key]), str.join(' ', dat2[key])))
    resfile.write("%s %s %s" % (key, str.join(' ', dat1[key]), str.join(' ', dat2[key])))

Answer 5

You can use:

doc1_name = 'doc1'
doc2_name = 'doc2'

def get_key_and_value(key_value_list):
    if len(key_value_list) == 2:
        # list has key and values
        key, value = key_value_list
    elif len(key_value_list) == 1:
        # list only has key
        key, value = key_value_list[0], ''
    else:
        # should not happen!
        key, value = '', ''
    return key,value

def join_dict(key, value, _dict, sep=' '):
    if key in _dict.keys():
        _dict[key] = sep.join((_dict[key], value))
    else:
        _dict[key] = value

result = {}
with open(doc1_name, 'r') as doc1, \
     open(doc2_name, 'r') as doc2:
         doc1_lines = doc1.readlines()
         doc2_lines = doc2.readlines()

for list_of_lines in (doc1_lines, doc2_lines):
    for line in list_of_lines:
        # The .strip('\n') removes the \n at the end
        # and the .split(' ', 1) split only once
        key_value = line.strip('\n').split(' ', 1)
        # split the lines only once to get the keys:
        key, value = get_key_and_value(key_value)
        # this can be ignored if it is known that the keys will be the same
        join_dict(key, value, result)

# order the keys
ordered_keys = result.keys()
ordered_keys.sort()
# and write them to a file
with open('+'.join((doc1_name,doc2_name)),'w') as output:
    for key in ordered_keys:
        output.write(' '.join((key, result[key]))+'\n')

doc1

1000001 10:0.471669 250:0.127552 30:0.218773 64:0.249413
1000002 130:0.0839656 107:0.185613 30:0.446355 110:0.38011
1000003 1:0.0835855 1117:0.0647112 302:0.0851354 46:0.0601825 48:0.098907 516:0.167713

doc2

1000001 161:0.115664 207:0.136537 294:0.0974809 301:0.199868
1000002
1000003 555:0.0585849 91:0.0164101

doc1+doc2

1000001 10:0.471669 250:0.127552 30:0.218773 64:0.249413 161:0.115664 207:0.136537 294:0.0974809 301:0.199868
1000002 130:0.0839656 107:0.185613 30:0.446355 110:0.38011 
1000003 1:0.0835855 1117:0.0647112 302:0.0851354 46:0.0601825 48:0.098907 516:0.167713 555:0.0585849 91:0.0164101