python 2 vs python 3 pd.merge命令

Question

The following command works fine if I run python 2

df5b = pd.merge(df5a, df5bb, how='outer')

However, when I run the same command with the same dfs in python 3, I get the following error:

ValueError: You are trying to merge on object and int64 columns. If you wish to proceed you should use pd.concat

My dataframes are very large, I hope that someone can help me out, without giving examples of my dataframes. The command is ok with python 2, so I assume that the problem are not the dataframes, but maybe a change of this command in python 3?

Answer 1

There is problem some columns are integers in one DataFrame and strings in another with same names.

Simpliest solution is cast all columns to strings:

df5b = pd.merge(df5a.astype(str), df5bb.astype(str), how='outer')

Another is test dtypes:

print (df5a.dtypes)
print (df5bb.dtypes)

And convert columns for same, eg convert strings columns from list to integers:

cols = ['col1','col12','col3']
df5a[cols] = df5a[cols].astype(int)

Sample :

df5a = pd.DataFrame({
         'B':[4,5,4,5],
         'C':[7,8,9,4],
         'F':list('aaab')
})

df5bb = pd.DataFrame({
         'B':['4','5','5'],
         'F':list('aab')
})

df5b = pd.merge(df5a.astype(str), df5bb.astype(str), how='outer')
print (df5b)

   B  C  F
0  4  7  a
1  4  9  a
2  5  8  a
3  5  4  b

---

print (df5a.dtypes)
B     int64
C     int64
F    object
dtype: object

print (df5bb.dtypes)
B    object
F    object
dtype: object

cols = ['B']
df5bb[cols] = df5bb[cols].astype(int)

df5b = pd.merge(df5a, df5bb, how='outer')
print (df5b)

   B  C  F
0  4  7  a
1  4  9  a
2  5  8  a
3  5  4  b

Answer 2

as i stated in my comment section, coerce is not happening on mixed types(may be int , str or float ) hence you can consider concat or you can convert them as str and then merge which jezrael mentioned.

Just to determine types you can see ..

>>> pd.concat([df5a, df5bb]).dtypes
B     object
C    float64
F     object
dtype: object

>>> pd.concat([df5a, df5bb])
   B    C  F
0  4  7.0  a
1  5  8.0  a
2  4  9.0  a
3  5  4.0  b
0  4  NaN  a
1  5  NaN  a
2  5  NaN  b