從 pickle 文件加載 spacy model 時出錯

Question

從 pickle 文件加載 spacy NER model 時出現以下錯誤。

self.model = pickle.load(open(model_path, 'rb')) 

Traceback (most recent call last):   
File "C:\Program Files\JetBrains\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py", line 1496, in _exec     
pydev_imports.execfile(file, globals, locals)  # execute the script   File "C:\Program Files\JetBrains\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile     
exec(compile(contents+"\n", file, 'exec'), glob, loc)   File 
"C:\Projects\pythonworkspace\invoice_processing_prototype\invoice_data_extractor_notebook.py", line 101, in 
<module>     
extractor = InvoiceDataExtractor(model_dir_path, input_file_paths[0], config_path)   File 
"C:\Projects\pythonworkspace\invoice_processing_prototype\invoicedataextractor.py", line 27, in 
__init__ self.spatial_extractor = SpatialExtractor(model_dir_path, config_path)   
File "C:\Projects\pythonworkspace\invoice_processing_prototype\spatialextractor.py", line 54, in __init__     
self.inv_date = Model(f{self.model_dir_path}\\invoice_date_with_corrected_training_data_and_line_seperator_21_07_2022.pkl",   File 
"C:\Projects\pythonworkspace\invoice_processing_prototype\spatialextractor.py", line 34, in __init__     

self.model = pickle.load(open(model_path, 'rb'))   
File "stringsource", line 6, in spacy.pipeline.trainable_pipe.__pyx_unpickle_TrainablePipe _pickle.PickleError: 
Incompatible checksums (0x417ddeb vs (0x61fbab5, 0x27e6ee8, 0xbe56bc9) = (cfg, model, name, scorer, vocab))

運行以下代碼行時出現此錯誤：

self.model = pickle.load(open(model_path, 'rb'))

我已經使用 spacy 版本 3.1.2 訓練了 NER model，最近我將 spacy 升級到了最新的 3.4。 該錯誤可能是因為某些版本不兼容。 如果是這種情況，有人可以確認是否可以加載在 spacy 版本“3.1.2”上訓練的 spacy NER model 可以在升級的 spacy“3.4”上加載

環境

操作系統：Windows 10

Python 使用的版本：3.10

訓練時使用的 spaCy 版本：3.1.2

預測時使用的 spaCy 版本：3.4

Answer 1

我找到了解決此錯誤的一種方法。 在spacy 3.1.2上，我加載了 pickle model 並使用保存

model = pickle.load(open(filepath, 'rb'))    
model.to_disc('my_model')

然后我將 spacy 版本更新到最新的spacy 3.4.4並這次重新加載了“my_model”。

model = spacy.load("my_model")
pickle.dump(model, open(filepath, 'wb'))  

在那種情況下，它對我有用而沒有錯誤。