Python中的复杂数据处理

Question

I have 3 files with real data and pseudo data and value of real data.

File_one has two columns with one column as real data and the second column as the translational data. Ie For the real data a pseudo value is given.

col[0] col[1]
123     0
234     1
345     2
456     3
567     4
678     5

File_two has pairs of pseudo values ie In place of 123 the value used is 0 and the same way the pseudo value pairs as [0, 1] which means [123, 234] in real.

col[0]  col[1]
0        2
0        3
0        5
2        4
5        1

So can say that col[0] and col[1] of file_two are the key and the value is in file_one col[0]

Now I have to match the pseudo value pairs from file_two with the real data col[0] in file_one and get an output saving it to new file. We name it file_four . Here pairs occur only ONE time.

col[0]  col[1]
123     345
123     456
123     678
345     567
678     234

Now file_three comes into the picture. File_three has the 3 columns.

col[0] and col[1] are the same pairs as in file_four but they also have many other pairs that are not present in file_four .

File_three

col[0]  col[1]  col[2]
123     345       54
345     262       65
123     456       54
2456    2467      98
123     678       46
7845    2458      631
345     567       153
3456    3673      94
678     234       5

Finally, I need to match the pairs of file_four ie col[0] col[1] and pull the value from col[2] in file_three and generate a new output_file with the pairs of file_four as key and the value in col[2] of file_three .

In the following code I am trying to only consider first two files

from collections import defaultdict

d1 = dict()
d2 = dict()

with open('input1.txt', 'r') as file1:
    for row in file1:
        c0, c1 = row.split()[:2]
        d1[c1] = c0
with open('input2.txt', 'r') as file2:
    for row in file2:
        c0, c1 = row.split()[:2]
        d2[(c0, c1)] = [d1[c1], d1[c1]]

#for k, v in sorted(d2.items()):
    #print '\t'.join(v)
print d2

Error:

Key Error: 'key' 

Its the same error even if for loop is not commented and last print is commented.

Answer 1

You don't have matching keys because d1 contains pairs as keys, while d2 contains single values.

This line looks like it is wrong:

    key =  col[0], col[1]

Answer 2

For d1 , use file1 column 1 for the keys and column 0 for the values creating a lookup table:

f1 = [(123,0),(234,1),(345,2),(456,3),(567,4),(678,5)]
f2 = [(0,2),(0,3),(0,5),(2,4),(5,1)]

d1 = {c1:c0 for c0,c1 in f1}

That allows you to use file2 column values to look up the values in d1

d2 = {(c0, c1):[d1[c0], d1[c1]] for c0, c1 in f2}
print d2

>>>
{(5, 1): [678, 234], (0, 3): [123, 456], (0, 5): [123, 678], (0, 2): [123, 345], (2, 4): [345, 567]}
>>>

Your code for file 1 and file 2 refactored :

d1, d2 = dict(), dict()
with open('inputfile1.txt', 'r') as file1:
    for row in file1:
        c0, c1 = row.strip().split()[:2]
        d1[c1] = c0

with open('inputfile2.txt', 'r') as file2:
    for row in file2:
        c0, c1 = row.strip().split()[:2]
        d2[(c0, c1)] = [d1[c0], d1[c1]]

>>> for k, v in sorted(d2.items()):
    print '\t'.join(v)


123 345
123 456
123 678
345 567
678 234
>>> 

Unpacking values/items during assignment:

>>> 
>>> x, y, z = [1, 2, 3]
>>> print x, y, z
1 2 3
>>> x, y = [1, 2, 3]

Traceback (most recent call last):
  File "<pyshell#259>", line 1, in <module>
    x, y = [1, 2, 3]
ValueError: too many values to unpack
>>> 
>>> a, b, _, _, _, _ = '1 2 3 4 5 6'.split()
>>> print a, b, _
1 2 6
>>>