带有在运行时传递的参数的装饰器

Question

There are many questions related to using decorators with or without parameters in python but my case is a bit different.

General Question

What I need is a decorator which accepts some parameters besides the function itself (and this functions args and kwargs). However these parameters are computed after the decorated function has been declared.

My specific case

I have written a number of functions processing a pd.DataFrame for machine learning, they get an DF as an input and return a DF as an output. In many cases I want to call them not on one DF, but on the concatenation of multiple training and test sets. The motivation is that I hate code duplication and the alternative is calling every step twice (once for each set). Also the author of a preprocessing function should not care about the fact that users often want to use it on 2 sets at a time.

The code currently looks like that:

def train_and_test(preprocess):
  def preprocess_wrapper(**kwargs):
    def concat(train, test):
        train['is_train'] = True
        test['is_train'] = False
        return pd.concat([train, test])

    def split(full):
        train = full[full['is_train']]
        test = full[~full['is_train']]
        train = train.drop('is_train', axis=1)
        test = test.drop('is_train', axis=1)
        assert len(full) == (len(train) + len(test))
        return train, test

    train = kwargs['train']
    test = kwargs['test']
    full = concat(train, test)
    processed = preprocess(full)
    return split(processed)

return preprocess_wrapper

@train_and_test # I would like to pass train and test as arguments here but these are loaded by client code
def my_preprocessor(df):
  preprocessed = do_something_smart(df)
  return preprocessed

What I want

I would like clients of this code to be able to call functions defined on one DF by passing the train and test sets to the decorator. Something like this:

train, test = pd.read_csv('data/train.csv'), pd.read_csv('data/test.csv')

train, test = preprocess(train, test) # In reality preprocess signature expects one DF but user can now treat it like it accepted 2

Answer 1

We need to go deeper!

A decorator is just a function that operates on a function.

Well, why not write a function that returns a function operating on a function?

def paramizer(*args):
    def normal_decorator(f):
        ...stuff that uses the decorator args to customize behavior...
        def wrapper():
            ...stuff decorating f...
            return f()
        return wrapper
    return normal_decorator

@paramizer(arg1, arg2) #=> returns a customized decorator and applies it to function
def function()