在python中减去匹配的行

Question

I got two files each containing a column with "time" and one with "id" like this:

File 1:

time     id
11.24    1
11.26    2
11.27    3
11.29    5
11.30    6

File 2:

time     id
11.25    1
11.26    3
11.27    4
11.31    6
11.32    7
11.33    8

Im trying to do a python script which can subtract the time of the rows with matching id from each other. The files are of different length.

I tried using set(id's of file 1) & set(id's of file 2) to get the matching id, but now I'm stuck. Any help will be much appreciated, thank you.

Answer 1

List comprehensions can do the trick very easily:

#read these from file if you want to, included in this form for brevity
F1 = {1: 11.24, 2: 11.26, 3:11.27, 5:11.29, 6:11.30}
F2 = {1:11.25, 3:11.26, 4:11.27, 6:11.31, 7:11.32, 8:11.33}

K1 = set(F1.keys())
K2 = set(F2.keys())

result = dict([ (k, F1[k] - F2[k]) for k in (K1 & K2)])
print result

This will output:

{1: -0.009999999999999787, 3: 0.009999999999999787, 6: -0.009999999999999787}

Edit: As mhawke points out, the last line could read:

result = {k: F1[k] - F2[k]) for k in (K1 & K2)}

I had forgotten all about dict comprehensions.

Answer 2

Python Set do not support ordering for the elements. I would store the data as a dictionary

file1 = {1:'11:24', 2:'11:26', ... etc}
file2 = {1:'11:25', 3:'11:26', ... etc}

The loop over the intersection of the keys (or union based on your needs) to do the subtraction (time based or math based).

Answer 3

This is a bit old school. Look at using a default dict from the collections module for a more elegant approach.

This will work for any number of files, I've named mine f1 , f2 etc. The general idea is to process each file and build up a list of time values for each id. After file processing, iterate over the dictionary subtracting each value as you go (via reduce on the values list).

from operator import sub

d = {}
for fname in ('f1','f2'):
    for l in open(fname):
        t, i = l.split()
        d[i] = d.get(i, []) + [float(t)]

results = {}
for k,v in d.items():
    results[k] = reduce(sub, v)

print results
{'1': -0.009999999999999787, '3': 0.009999999999999787, '2': 11.26, '5': 11.29, '4': 11.27, '7': 11.32, '6': -0.009999999999999787, '8': 11.33}

Updated

If you want to include only those ids with more than one value:

results = {}
for k,v in d.items():
    if len(v) > 1:
        results[k] = reduce(sub, v)

Answer 4

You can use this as a base (instead of treating '11.24' as a float, I guess you want to adapt for hours/minutes or minutes/seconds)... you can effectively union and subtract matching keys using a defaultdict .

As long as you can get your data into a format like this:

f1 = [
    [11.24, 1],
    [11.26, 2],
    [11.27, 3],
    [11.29, 5],
    [11.30, 6]
]

f2 = [
    [11.25, 1],
    [11.26, 3],
    [11.27, 4],
    [11.31, 6],
    [11.32, 7],
    [11.33, 8]
]

Then:

from collections import defaultdict
from itertools import chain

dd = defaultdict(float)
for k, v in chain(
    ((b, a) for a, b in f1),
    ((b, -a) for a, b in f2)): # negate a

    dd[k] += v

Results in:

{1: -0.009999999999999787,
 2: 11.26,
 3: 0.009999999999999787,
 4: -11.27,
 5: 11.29,
 6: -0.009999999999999787,
 7: -11.32,
 8: -11.33}

For matches only

matches = dict( (k, v) for v, k in f1 )
d2 = dict( (k, v) for v, k in f2 )

for k, v in matches.items():
    try:
        matches[k] = v - d2[k]
    except KeyError as e:
        del matches[k]

print matches
# {1: -0.009999999999999787, 3: 0.009999999999999787, 6: -0.009999999999999787}