I'm aware about the replace function in pandas: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.replace.html
But I've done this simple test and it is not working as expected when I try to replace a float value:
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randn(50, 4), columns=list('ABCD'))
print(df.head(n=1))
A B C D
0 1.437202 1.919894 -1.40674 -0.316737
df = df.replace(1.437202, np.nan)
print(df.head(n=1))
A B C D
0 1.437202 1.919894 -1.40674 -0.316737
As you see the [[0],[0]] has no change...any idea about what this could be due to?
Problem is float precision, so use function numpy.isclose
with mask
:
np.random.seed(123)
df = pd.DataFrame(np.random.randn(50, 4), columns=list('ABCD'))
print(df.head(n=1))
A B C D
0 -1.085631 0.997345 0.282978 -1.506295
df = df.mask(np.isclose(df.values, 0.997345))
Or use numpy.where
:
arr = np.where(np.isclose(df.values, 0.997345), np.nan, df.values)
df = pd.DataFrame(arr, index=df.index, columns=df.columns)
print(df.head(n=1))
A B C D
0 -1.085631 NaN 0.282978 -1.506295
EDIT: You can also get only numeric columns by select_dtypes
for filtering by subset with []
:
np.random.seed(123)
df = pd.DataFrame(np.random.randn(50, 4), columns=list('ABCD')).assign(E='a')
cols = df.select_dtypes(np.number).columns
df[cols] = df[cols].mask(np.isclose(df[cols].values, 0.997345))
print(df.head(n=1))
A B C D E
0 -1.085631 NaN 0.282978 -1.506295 a
Just a another trick for specific indices :
>>> print(df.head(n=1))
A B C D
0 -0.042839 1.701118 0.064779 1.513046
>>> df['A'][0] = np.nan
>>> print(df.head(n=1))
A B C D
0 NaN 1.701118 0.064779 1.513046
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.