ColumnTransformer fit_transform 不适用于管道

Question

I am writing a pipeline with custom transformer. When calling fit_transform of categorical pipeline I am getting the desired result but when calling fit_transform of ColumnTransformer, whatever I have initialised in init of custom transformer is getting lost. Note: not including code of numericalTransformer for readability

class categoryTransformer(BaseEstimator, TransformerMixin):
def __init__(self, use_dates=['year', 'month', 'day']):
    self._use_dates = use_dates
    print('==========>',self._use_dates)
def fit(self, X, y=None):
    return self

def get_year(self, obj):
    return str(obj)[:4]

def get_month(self, obj):
    return str(obj)[4:6]

def get_day(self, obj):
    return str(obj)[6:8]

def create_boolean(self, obj):
    if obj == '0':
        return 'No'
    else:
        return 'Yes'
    
def transform(self, X, y=None):
    print(self._use_dates)

     for spec in self._use_dates:
         print(spec)
         exec("X.loc[:,'{}'] = X['date'].apply(self.get_{})".format(spec, spec))
    
    X = X.drop('date', axis=1)
    X.loc[:,'yr_renovated'] = X['yr_renovated'].apply(self.create_boolean)
    X.loc[:, 'view'] = X['view'].apply(self.create_boolean)
    return X.values

cat_pipe = Pipeline([
('cat_transform', categoryTransformer()),
('one_hot', OneHotEncoder(sparse=False))])

num_pipe = Pipeline([
('num_transform', numericalTransformer()),
('imputer', SimpleImputer(strategy = 'median')),
('std_scaler', StandardScaler())])

full_pipe = ColumnTransformer([
('num', num_pipe, numerical_features),
('cat', cat_pipe, categorical_features)])

cat_pipe.fit_transform(data[categorical_features])#working fine
df2 = full_pipe.fit_transform(X_train)# __init__ initialisation lost

"output"
==========> ['year', 'month', 'day']
['year', 'month', 'day']
year
month
day
==========> None
None

After that long traceback that I am not able to debug. Workaround is if I can create use_dates=['year', 'month', 'day'] in transform function itself but I want to understand why this is happening.

Answer 1

The parameters of __init__ need to have the same names as the attributes that get set (so use_dates and _use_dates is the problem).

This is required for cloning to work properly, and ColumnTransformer clones all its transformers before fitting.

https://scikit-learn.org/stable/developers/develop.html#instantiation