How to solve Dataframe to_numeric Error (Python)?

Question

I have dataframe that contains float numbers. I want to do numeric operations as sum mult. etc. . Columns types are object. So i have to change this columns into a numeric columns. I use to_numeric function but it gives me NaN as a result.

How can i solve this problem?

Code:

#import libraries
import pandas as pd

data = pd.read_csv('file.csv', engine='python', delimiter=';')

#change object columns into a numeric columnn
for i in data.columns :
    data[i] = pd.to_numeric(data[i], errors='coerce')

Dataframe

t0 (actual) t0  t0,lower    t0,upper
0   11861,6318726842    0   0   0
1   4761,43316  5709,1728515625 3776,725188260803   7939,908970830105
2   36,22841951973635   0   0   0
3   583,3716479196096   0   0   0
4   25087,16436661841   26040,7890625   21825,20941707611   31905,394350044822
....

Result:

    t0 (actual) t0  t0,lower    t0,upper
0   NaN NaN NaN NaN
1   NaN NaN NaN NaN
2   NaN NaN NaN NaN
3   NaN NaN NaN NaN
4   NaN NaN NaN NaN

Answer 1

I can't reproduce your error, but I assume you could add decimal="," to pd.read_csv .

From the docs :

decimal: str, default '.'
Character to recognize as decimal point (eg use ',' for European data).

So your code would look like this:

#import libraries
import pandas as pd

data = pd.read_csv('file.csv', engine='python', delimiter=';', decimal=",")

#change object columns into a numeric columnn
for i in data.columns :
    data[i] = pd.to_numeric(data[i], errors='coerce')

Answer 2

If you change errors from coerce to raise , you will see that pandas cannot convert these values to numerical datatype. This is because it does not recognize , as a decimal separator (by default, it is . ). Which means having errors = 'coerce' will replace these values with NaN .

If, as it seems, you're trying to convert these values right after reading it from a .csv file, you should instead specify the decimal separator used in your read_csv call. Then you will not need to convert the values manually.

data = pd.read_csv('file.csv', engine='python', delimiter=';', decimal = ',')

If, however, these values come from your script, you would need to replace , values with . in your string values:

for i in data.columns :
    data[i] = pd.to_numeric(data[i].str.replace(',','.'), errors='coerce')