How to perform reconciliation using DataFrame?

Question

Let's say I have a DataFrame that I get weekly and would like to update a tracker that is updated based on a weekly report, for example:

This is a weekly report I receive:

ID   Cost
X12  500
X54  100
X52  150
X45  200
X32  435

I have a DataFrame for the main metric tracker, that needs to be updated based on the weekly report:

ID   Cost
X12  34
X54  467
X52  234
X45  3453
X37  4664    
X76  34
X57  467
X52  23465
X48  547
X32  34

I would like to take the numbers from DataFrame 1 and put them into DataFrame 2, which would result in:

ID   Cost
X12  500
X54  100
X52  150
X45  200
X37  4664    
X76  34
X57  467
X56  23465
X48  547
X32  435

How would I go about performing a function like this, from one DataFrame onto another. I assume we use a for loop and search through the second DataFrame while iterating through the first, but how would I set this up?

Thanks!

Answer 1

If order isn't an issue, then

In [1178]: dfw.set_index('ID').combine_first(dfo.set_index('ID')).reset_index()
Out[1178]:
    ID    Cost
0  X12   500.0
1  X32   435.0
2  X37  4664.0
3  X45   200.0
4  X48   547.0
5  X52   150.0
6  X52   150.0
7  X54   100.0
8  X57   467.0
9  X76    34.0

This will take care of any new ID additions as well.

Answer 2

You can use map and combine_first

df2['Cost'] = df2['ID'].map(df1.set_index('ID')['Cost']).combine_first(df2['Cost'])

You get

    ID  Cost
0   X12 500.0
1   X54 100.0
2   X52 150.0
3   X45 200.0
4   X37 4664.0
5   X76 34.0
6   X57 467.0
7   X52 150.0
8   X48 547.0
9   X32 435.0

Answer 3

You can do like this.

where 
df -> first dataframe
df1 -> second dataframe

Code:

df.loc[df.ID.isin(df1.ID), ['Cost']] = df1[['Cost']]
print df

Reference:

Replace column values based on another dataframe

Answer 4

Using drop_duplicates

rep.append(track).drop_duplicates('ID')

    ID  Cost
0  X12   500
1  X54   100
2  X52   150
3  X45   200
4  X32   435
4  X37  4664
5  X76    34
6  X57   467
8  X48   547

---

If you want to reset your index

rep.append(track).drop_duplicates('ID').reset_index(drop=True)

    ID  Cost
0  X12   500
1  X54   100
2  X52   150
3  X45   200
4  X32   435
5  X37  4664
6  X76    34
7  X57   467
8  X48   547