Pandas dataframe left-merge with different dataframe sizes

Question

I have a toy stock predictor, and from time to time save results using dataframes. After the first result set I would like to append my first dataframe. Here is what I do:

1. Create first dataframe using predicted results
2. Sort descending to predicted performance

3. Save to csv, without the index

4. With new data, read out result csv and try left merge, goal is to append new predicted performance to the correct stock ticker

df=pd.merge(df, df_new[['ticker', 'avgrd_app']], on='ticker', how='left')

Those two dataframes have different amounts of columns. In the end it only appends the dataframes to another:

avgrd,avgrd_app,prediction1,prediction2,ticker

-0.533520756811,,110.64654541,110.37853241,KIO
-0.533520756811,,110.64654541,110.37853241,MMM
-0.604610694122,,110.64654541,110.37853241,SRI  
[...]
,-0.212600450514,,,G5DN
,0.96378750992,,,G5N
,2.92757501984,,,DAL3
,2.27297945023,,,WHF4

So - how can I merge correctly?

Answer 1

From the sample result, it works as expected, the new data don't have numbers for all the tickers so some of the predictions are missing. So what exactly do you want to achieve? If you only need stocks with all the predictions, use inner join.