Python Pandas: Append Dataframe To Another Dataframe Only If Column Value is Unique

Question

I have two data frames that I want to append together. Below are samples.

df_1:

Code    Title
103     general checks 
107     limits
421     horseshoe
319     scheduled 
501     zonal 

df_2

Code    Title
103     hello 
108     lucky eight 
421     little toe 
319     scheduled cat
503     new item 

I want to append df_2 to df_1 ONLY IF the code number in df_2 does not exist already in df_1.

Below is the dataframe I want:

Code    Title
103     general checks 
107     limits
421     horseshoe
319     scheduled 
501     zonal 
108     lucky eight 
503     new item

I have searched through Google and Stackoverflow but couldn't find anything on this specific case.

Answer 1

Just append the filtered data frame

df3 = df2.loc[~df2.Code.isin(df.Code)]
df.append(df3)

    Code    Title
0   103 general checks
1   107 limits
2   421 horseshoe
3   319 scheduled
4   501 zonal
1   108 lucky eight
4   503 new item

Notice that you might end up with duplicated indexes, which may cause problems. To avoid that, you can .reset_index(drop=True) to get a fresh df with no duplicated indexes.

df.append(df3).reset_index(drop=True)

    Code    Title
0   103 general checks
1   107 limits
2   421 horseshoe
3   319 scheduled
4   501 zonal
5   108 lucky eight
6   503 new item

Answer 2

You can concat and then drop_duplicates . Assumes within each dataframe Code is unique.

res = pd.concat([df1, df2]).drop_duplicates('Code')

print(res)

   Code           Title
0   103  general_checks
1   107          limits
2   421       horseshoe
3   319       scheduled
4   501           zonal
1   108     lucky_eight
4   503        new_item

Answer 3

Similar to concat(), you could also use merge:

df3 = pd.merge(df_1, df_2, how='outer').drop_duplicates('Code')

    Code    Title
0   103 general checks
1   107 limits
2   421 horseshoe
3   319 scheduled
4   501 zonal
6   108 lucky eight
9   503 new item