为什么在 Python object 被腌制并重新加载后，某些属性会丢失？

Question

在我将实例转储到 pickle 文件并将其加载回来后，我遇到了一个实例的某些属性丢失的问题。 谁能帮忙解释一下？ 谢谢！

这是一个具体的例子：

File/directory hierachy:
-test
 -test_module
  -__init__.py
  -myDataFrameMapper.py
  -mySklearn.py
 -main.py

__init__.py：

from .mySklearn import mySklearn

我的Sklearn.py

import sklearn_pandas as sk_pd
from .myDataFrameMapper import myDataFrameMapper

class mySklearn:
      def initialize():
          sk_pd.DataFrameMapper.myTransform = myDataFrameMapper.transform()

myDataFrameMapper.py

import numpy as np
from sklearn_pandas import DataFrameMapper

class myDataFrameMapper:

      def transform():
          def closure(self, df, **kwargs):
                 self.addedKey = 'addedValue' # a new attribute is added here
          return closure

主文件

import pandas as pd
import pickle
import random
from sklearn_pandas import DataFrameMapper
from sklearn.preprocessing import StandardScaler, LabelEncoder

from test_module import mySklearn

mySklearn.initialize()

data = {'pet':["cat", "dog", "dog", "fish", "cat", "dog", "cat", "fish"],
        'children':[4., 6, 3, 3, 2, 3, 5, 4],
        'salary':[90, 24, 44, 27, 32, 59, 36, 27]}

df = pd.DataFrame(data)

column_tuples = [
   ('pet', LabelEncoder()),
   ('children', LabelEncoder()),
   ('salary', LabelEncoder())
]

mapper = DataFrameMapper(column_tuples, input_df=True)
mapper.fit(data)

print('original attributes in mapper:')
print(mapper.__dict__)

mapper.myTransform(df.iloc[[1]])

print('\nafter adding a new attributes \'addedKey\':')
print(mapper.__dict__)

print('\ndump the mapper into a pickle file...')
picklefile = open('mapper.pkl', 'wb')
pickle.dump(mapper, picklefile)
picklefile.close()

print('\nload the mapper from the pickle file...')
picklefile = open('mapper.pkl', 'rb')
mapper1 = pickle.load(picklefile)
picklefile.close()

print('\nafter being loaded, the attributes in the mapper are:')
print(mapper1.__dict__)

运行python3 main.py后，我们观察到以下输出：

original attributes in mapper:
{'built_default': False, 'sparse': False, 'input_df': True, 'df_out': False, 'features': [('pet', LabelEncoder()), ('children', LabelEncoder()), ('salary', LabelEncoder())], 'default': False, 'built_features': [('pet', LabelEncoder(), {}), ('children', LabelEncoder(), {}), ('salary', LabelEncoder(), {})], 'transformed_names_': []}

after adding a new attributes 'addedKey':
{'built_default': False, 'addedKey': 'addedValue', 'sparse': False, 'input_df': True, 'df_out': False, 'features': [('pet', LabelEncoder()), ('children', LabelEncoder()), ('salary', LabelEncoder())], 'default': False, 'built_features': [('pet', LabelEncoder(), {}), ('children', LabelEncoder(), {}), ('salary', LabelEncoder(), {})], 'transformed_names_': []}

dump the mapper into a pickler file:

load the mapper from the pickle file:

after being loaded, the attributes in the mapper are:
{'built_default': False, 'sparse': False, 'input_df': True, 'df_out': False, 'features': [('pet', LabelEncoder(), {}), ('children', LabelEncoder(), {}), ('salary', LabelEncoder(), {})], 'default': False, 'built_features': [('pet', LabelEncoder(), {}), ('children', LabelEncoder(), {}), ('salary', LabelEncoder(), {})], 'transformed_names_': []}

我们可以看到，当映射器从 pickle 文件加载回来时，属性'addedKey': 'addedValue'丢失了。

Answer 1

sklearn_pandas.DataFrameMapper有一个自定义的__setstate__方法，试图保持 pickle 与旧版本上创建的 pickle 的兼容性。 （ 这是该方法的 1.8.0 版本。）此__setstate__负责恢复未腌制实例的 state，它完全忽略了您添加的属性。

Pickle 实现自定义是尝试将自己的属性添加到其他人的类通常是一个坏主意的原因之一。