Pandas how to fillna in place on a column?

Question

After running:

df[['column']].fillna(value=myValue, inplace=True)

or:

df['column'].fillna(value=myValue, inplace=True)

or:

# Throws warning "A value is trying to be set on a copy of a slice..."
df.fillna({'column': myValue}, inplace=True)

or:

df[['column']] = df[['column']].fillna({'column': myValue})

or:

df['column'] = df['column'].fillna({'column': myValue})

My df['column'] still contains nan (!)

list(df['column'].unique()) returns ['a', 'b', 'c', 'd', nan] and sum(pd.isnull(df['column'])) returns 1,000+.

I've tried several variations but this problem persists. How do you fillna in place on a column in pandas?

Answer 1

Ed Chum's comment's correctly points out the difference between the methods you propoosed. Here is an example I used to show how it works.

import pandas as pd
import numpy as np

d = {'col1': [1, 2, 3, 4], 'col2': [3, 4, np.nan, np.nan]}
df = pd.DataFrame(data=d)

df

   col1  col2
0     1   3.0
1     2   4.0
2     3   NaN
3     4   NaN

df['col2'].fillna(value=6, inplace=True)

   col1  col2
0     1   3.0
1     2   4.0
2     3   6.0
3     4   6.0

Having posted this, I think it'd be most valuable to see what your my_value variable's value is and what your dataframe looks like.

I discard Aditya's hypothesis. In the case the nan would be a string, it would appear between quotations marks, and it doesn't.

Hope this helps!

Answer 2

One cause of this problem can be that the nan values in your dataset might be the string 'nan' instead of NaN. To solve this, you can use the replace() method instead of fillna().

Eg code:

df['column'].replace(to_replace='nan',value=myValue,inplace=True)

Answer 3

First of all, the correct syntax from your list is

df['column'].fillna(value=myValue, inplace=True)

If list(df['column'].unique()) returns ['a', 'b', 'c', 'd', nan] , this means that the values in your dataset are probably not equal to np.NaN , but rather equal to the string "nan".