[英]Python: TypeError: Pickling an AuthenticationString object is disallowed for security reasons
I'm creating an object of a class(with multiprocessing
) and adding it to a Manager.dict()
so that I can delete the item from the dictionary inside the object (the item points to) when its work completes..我正在创建一个类的对象(使用
multiprocessing
)并将其添加到Manager.dict()
以便我可以在其工作完成时从对象内部的字典中删除该项目(该项目指向)。
I tried the following code:我尝试了以下代码:
from multiprocessing import Manager, Process
class My_class(Process):
def __init__(self):
super(My_class, self).__init__()
print "Object", self, "created."
def run(self):
print "Object", self, "process started."
manager=Manager()
object_dict=manager.dict()
for x in range(2):
object_dict[x]=My_class()
object_dict[x].start()
But I got an error:但我得到了一个错误:
TypeError: Pickling an AuthenticationString object is disallowed
for security reasons
For curiosity, I removed the multiprocessing part, and tried like:出于好奇,我删除了多处理部分,并尝试如下:
from multiprocessing import Manager
class My_class():
def __init__(self):
print "Object", self, "created."
manager=Manager()
object_dict=manager.dict()
for x in range(2):
object_dict[x]=My_class()
and it's giving me no errors and displaying the addresses of two objects.它没有给我任何错误并显示两个对象的地址。
What's that error and how to make it go away?这个错误是什么以及如何让它消失?
Here is a shorter way to replicate the effect you are seeing:这是复制您看到的效果的更短方法:
from multiprocessing import Process
import pickle
p = Process()
pickle.dumps(p._config['authkey'])
TypeError: Pickling an AuthenticationString object is disallowed for security reasons
类型错误:出于安全原因,不允许酸洗 AuthenticationString 对象
What is actually happening here is the following: the process._config['authkey']
is the secret key that the Process
object gets assigned on creation.这里实际发生的情况如下:
process._config['authkey']
是Process
对象在创建时分配的密钥。 Although this key is nothing more but a sequence of bytes, Python uses a special subclass of bytes
to represent it: AuthenticationString
.虽然这个键不过是一个字节序列,但 Python 使用了一个特殊的
bytes
子类来表示它: AuthenticationString
。 This subclass differs from the usual bytes
in only one aspect - it refuses to be pickled.这个子类与通常的
bytes
只有一个方面不同——它拒绝被腌制。
The rationale behind this choice is the following: the authkey is used for authenticating inter-process communication messages between parent and child processes (eg between the workers and the main process) and exposing it anywhere outside the initial process family could pose a security risk (because you could, in principle, impersonate a "parent process" for the worker and force it into executing arbitrary code).这种选择背后的基本原理如下:authkey 用于验证父进程和子进程之间(例如工作进程和主进程之间)的进程间通信消息,并将其暴露在初始进程家族之外的任何地方可能会带来安全风险(因为原则上您可以为工作人员模拟一个“父进程”并强制它执行任意代码)。 As pickling is the most common form of data transfer in Python, prohibiting it is a simple way of an unintended exposure of the authkey.
由于酸洗是 Python 中最常见的数据传输形式,因此禁止它是意外暴露 authkey 的一种简单方法。
As you cannot pickle an AuthenticationString
, you also cannot pickle instances of Process
class or any of its subclasses (because all of them contain an authentication key in a field).由于您无法选择
AuthenticationString
,因此您也无法选择Process
类或其任何子类的实例(因为它们都在字段中包含身份验证密钥)。
Now let us take a look at how it all relates to your code.现在让我们看看这一切与您的代码有何关联。 You create a
Manager
object and attempt to set the values of its dict
.您创建一个
Manager
对象并尝试设置其dict
的值。 The Manager
actually runs in a separate process and whenever you assign any data to manager.dict()
, Python needs to transfer this data to the Manager's
own process. Manager
实际上运行在一个单独的进程中,每当您将任何数据分配给manager.dict()
,Python 都需要将此数据传输到Manager's
自己的进程。 For that transfer to happen, the data is being pickled.为了进行这种传输,数据正在被腌制。 But, as we know from the previous paragraphs, you cannot pickle
Process
objects and hence cannot keep them in a shared dict
at all.但是,正如我们从前面的段落中知道的那样,您无法pickle
Process
对象,因此根本无法将它们保存在共享dict
中。
In short, you are free to use manager.dict()
to share any objects, except those which cannot be pickled, such as the Process
objects.简而言之,你可以自由地使用
manager.dict()
来共享任何对象,除了那些不能被腌制的对象,比如Process
对象。
Note: the solution below is in Python3 aka print() .注意:下面的解决方案是在 Python3 aka print() 中。 The same issue exists in Python3 also.
Python3 中也存在同样的问题。
Well, in your specific example, we can work around the problem by pickling the AuthenticationString
inside the _config
dict that's part of the Process
object as a bytes buffer and then gracefully restoring it when unpickling as if nothing happened.那么,在你的具体的例子,我们可以通过酸洗解决该问题
AuthenticationString
里面_config
字典是这样的部分Process
对象作为一个字节的缓冲区,然后优雅地取储存时,仿佛什么都没有发生还原它。 Define the get and set state methods that are called for pickling ops as follows inside My_class
:在
My_class
定义为酸洗操作调用的 get 和 set 状态方法,如下所示:
from multiprocessing import Manager, Process
from multiprocessing.process import AuthenticationString
class My_class(Process):
def __init__(self):
super(My_class, self).__init__()
print("Object", self, "created.")
def run(self):
print("Object", self, "process started.")
def __getstate__(self):
"""called when pickling - this hack allows subprocesses to
be spawned without the AuthenticationString raising an error"""
state = self.__dict__.copy()
conf = state['_config']
if 'authkey' in conf:
#del conf['authkey']
conf['authkey'] = bytes(conf['authkey'])
return state
def __setstate__(self, state):
"""for unpickling"""
state['_config']['authkey'] = AuthenticationString(state['_config']['authkey'])
self.__dict__.update(state)
if __name__ == '__main__': # had to add this
manager=Manager()
object_dict=manager.dict()
for x in range(2):
object_dict[x]=My_class()
object_dict[x].start()
I get the following output from running the code:我通过运行代码得到以下输出:
Object <My_class(My_class-2, initial)> created.
Object <My_class(My_class-3, initial)> created.
Object <My_class(My_class-2, started)> process started.
Object <My_class(My_class-3, started)> process started.
Which appears to be the intended outcome, and if you put a time.sleep()
call in to keep them alive a bit longer, you can see the two sub-processess running.这似乎是预期的结果,如果你
time.sleep()
来让它们活得更久,你可以看到两个子进程正在运行。
Alternatively, it doesn't seem to upset anything if you simply delete that _config
authkey
and then you don't even need to define a custom __setstate__
method.或者,如果您简单地删除该
_config
authkey
并且您甚至不需要定义自定义__setstate__
方法,它似乎也不会打扰任何事情。
Also, note that I had to add in __main__
- without it python complained about not having finished its bootstrapping before launching sub processes.另外,请注意,我必须添加
__main__
- 没有它,python 抱怨在启动子进程之前没有完成引导。
Finally, I just have to shrug my shoulders at this whole "security" thing.最后,我只需要对这整个“安全”事情耸耸肩。 It pops up in all sorts of places (with the same type of work-around required) and doesn't provide any real security.
它会出现在各种地方(需要相同类型的变通方法)并且不提供任何真正的安全性。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.