Pandas Dataframe and Series join returns empty Dataframe or NaN column

Question

The issue that I am having is that when I try to join a Dataframe and a Series, for some reason the resulting Dataframe is either empty or has a column of NaN values. I am trying to figure out why this is happening.

The Series looks like this:

index
110     0.135135
111     0.000000
1110    0.000000

The Dataframe looks like this:

           mean         std
index                        
1101     -41.000000   46.305225
1102     -58.724998  126.810371
1110      -6.375000   12.915982

When I don't specify 'how', I get this:

          mean        std             series_col
index                                 
1101     -41.000000   46.305225       NaN
1102     -58.724998   126.810371      NaN
1110      -6.375000   12.915982       NaN

This is how I tried to join the two using how:

merged = df1.join(series1, how='inner')

I get this output:

Empty DataFrame
Columns: [mean, std, series_col]
Index: []

I can't figure out what's wrong. I think it must be an index issue, but I know for certain that both the Dataframe and Series have some intersecting indices on which to match the data.

Let me know if there's any other useful information I can provide.

Answer 1

When I did this I got

df1.join(series1)

           mean         std    1
1101 -41.000000   46.305225  NaN
1102 -58.724998  126.810371  NaN
1110  -6.375000   12.915982  0.0

I'm guessing one of your indices are strings and the other are ints

Answer 2

There is problem indexes are not same dtypes , so get NaN .

Solution is cast both indexes to int or both to str for align:

series1.index = series1.index.astype(int)
df1.index = df1.index.astype(int)

---

series1.index = series1.index.astype(str)
df1.index = df1.index.astype(str)

For me it return:

#inner join
merged = df1.join(series1, how='inner')
print (merged)
        mean        std  val
index                       
1110  -6.375  12.915982  0.0

---

#default left join
merged = df1.join(series1)
#same as:  
merged = df1.join(series1, how='left')
print (merged)
            mean         std  val
index                            
1101  -41.000000   46.305225  NaN
1102  -58.724998  126.810371  NaN
1110   -6.375000   12.915982  0.0

---

merged = df1.join(series1, how='outer')
print (merged)
            mean         std       val
index                                 
110          NaN         NaN  0.135135
1101  -41.000000   46.305225       NaN
1102  -58.724998  126.810371       NaN
111          NaN         NaN  0.000000
1110   -6.375000   12.915982  0.000000

---

merged = df1.join(series1, how='right')
print (merged)
        mean        std       val
index                            
110      NaN        NaN  0.135135
111      NaN        NaN  0.000000
1110  -6.375  12.915982  0.000000