如何不使用 pandas 切割 function 来估算 NaN 值？

Question

I'm trying to use the cut function to convert numeric data into categories. My input data may have NaN values, which I would like to stay NaN after the cut. From what I understand reading the documentation, this is the default behavior and the following code should work:

intervals = [(i, i+1) for i in range(101)]
bins = pd.IntervalIndex.from_tuples(intervals)
pd.cut(pd.Series([np.nan,0.5,10]),bins)

However, the output I get is:

>(49,50]
 (0,1]
 (9,10]

Notice that the NaN value is converted to the middle interval.

One strange thing is that it appears as though once the number of intervals is 100 or less, I get the desired output:

intervals = [(i, i+1) for i in range(100)]
bins = pd.IntervalIndex.from_tuples(intervals)
pd.cut(pd.Series([np.nan,0.5,10]),bins)

output:

>NaN
 (0,1]
 (9,10]

Is there a way to specify that I don't want NaN values to be imputed?

Answer 1

This seems like a bug that originates from numpy.searchsorted() :

• pandas-dev/pandas#31586 - pd.cut returning incorrect output in some cases
• numpy/numpy#15499 - BUG: searchsorted with object arrays containing nan

As a workaround, you could replace np.nan with some other guaranteed missing value, eg .replace(np.nan,'foo') :

intervals = [(i, i+1) for i in range(101)]
bins = pd.IntervalIndex.from_tuples(intervals)
pd.cut(pd.Series([np.nan,0.5,10]).replace(np.nan,'foo'),bins)

0            NaN
1     (0.0, 1.0]
2    (9.0, 10.0]
dtype: category