Pandas rename column of dataframe to value of another dataframe if values of two dataframe columns match

Question

I have two dataframes.

dfA contains two columns "CCLE_ID" and "Name" amongst other unimportant ones

dfB contains two columns "CCLE ID" and "Cell line" amongst other unimportant ones.

Right now, dfB['CCLE ID'] values are set to 0.

What I want to do is compare all the values in dfA['Name'] column and dfB['Cell line'] column. They are all strings and stand for the shorthand name of cell lines. If a value for dfA['Name'] and dfB['Cell line'] column matches, then I want to replace the value 0 of dfB['CCLE ID'] column with the string from dfA['CCLE_ID'] column of that matched cell name.

I am honestly so lost as to how to do this (pandas beginner).

Answer 1

First we presume dfA and dfB have the same number of rows because if they don't, then it's more complicated and you have two choices: either reshape the dataFrames to have the same number of rows, or use other Python libraries to perform the transformation.

Based on this initial presumption that the data Frames have the same number of rows, I'm going to try and break this down for you step by step.

With the two dataframes, dfA and dfB , start by merging the data. You can remove the extra columns from dfB later.

To merge the dfA columns into dfB for simplicity, add two columns dfaName and dfa_CCLE_ID.

dfB['dfaName'] = dfa['Name']
dfB['dfa_CCLE_ID'] = dfa['CCLE_ID']

Then use pandas.dataFrame.apply() to conditionnally transform your data.

dfB['CCLE_ID'] = dfB[['dfaName','Cell line', 'dfa_CCLE_ID']].apply(lambda x: x['dfa_CCLE_ID'] if x['dfaName']==x['Cell line'] else x, axis=1)

A nice extra could be to use a dataframe mask to generate and see comparison. It is a good step to take to view and test your data transformation. In this example, create an extra column in dfB with true/false values for the comparison.

dfB['column_matcher'] = dfb['dfaName']==dfB['Cell line']