如何将 sklearn 预处理器 fit_transform 与 pandas.groupby.transform 一起使用

Question

How to use sklearn preprocessing fit.transform() with pandas.groupby.transform?

I used this code here that works:

Picture of sample dataframe

df.groupby('Category')['X1'].transform(lambda x: minmax_scale(x.astype(float)))

But when I changed it to the MinMaxScaler() method below, it returns error

Code with Error when using .fit_transform method

Assume the table only has 2 columns: Category and X1

df.groupby('Category')['X1'].transform(lambda x: MinMaxScaler().fit_transform(x.values.reshape(-1,1)))

Error Message:

Data must be 1-dimensional

However, if I don't use the .values.reshape(-1,1) it will say

Expected 2D array, got 1D array instead. Reshape your data either using array.reshape(-1, 1) if your data has a single feature

Are we not supposed to use the fit_transform method for .apply / .transform on pandas?

Edit: updated with new error message

Answer 1

You got to use MinMaxScaler object instance (add parenthses). Try this:

lambda x: MinMaxScaler().fit_transform(x.values.reshape(-1,1))

if you want to pass the scaling range, pass it to the constructor:

lambda x: MinMaxScaler(feature_range=(0, 10)).fit_transform(x.values.reshape(-1,1))

here is a working example:

df = pd.DataFrame (np.random.randint(1,100,(10)),columns = ['a'])
df['a'].transform(lambda x: MinMaxScaler(feature_range=(0, 10)).
                  fit_transform(x.values.reshape(-1,1)))

array([[ 0.        ],
       [ 6.55172414],
       [ 9.88505747],
       [ 6.09195402],
       [ 1.26436782],
       [ 8.62068966],
       [ 6.43678161],
       [ 5.74712644],
       [ 5.17241379],
       [10.        ]])

Answer 2

I just found the solution, which is to wrap the scaler with np.concatenate() Solution is similar to this thread here: Pandas groupby in combination with sklean preprocessing continued

So the working code looks like this:

df.groupby('Category')['X1'].transform(
lambda x: np.concatenate(StandardScaler().fit_transform(x.values.reshape(-1,1))))