[英]How might I persuade Kombu to use simplejson instead of the json module?
Is there a good way to persuade Celery+Kombu to use simplejson instead of the standard library's json module?有什么好的方法可以说服Celery+Kombu 使用simplejson 而不是标准库的json 模块吗?
I'm on:我上线了:
tact@tact_pub_api:/app$ python3.10 -m pip list -v | egrep -i 'celery|kombu'
celery 5.2.3 /usr/local/lib/python3.10/site-packages pip
kombu 5.2.4 /usr/local/lib/python3.10/site-packages pip
tact@tact_pub_api:/app$ python2.7 -m pip list -v | egrep -i 'celery|kombu'
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
celery 3.1.26.post2 /usr/local/lib/python2.7/dist-packages pip
kombu 3.0.37 /usr/local/lib/python2.7/dist-packages pip
...and I'm getting a traceback: ...我得到了回溯:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/kombu/serialization.py", line 39, in _reraise_errors
yield
File "/usr/local/lib/python3.10/site-packages/kombu/serialization.py", line 210, in dumps
payload = encoder(data)
File "/usr/local/lib/python3.10/site-packages/kombu/utils/json.py", line 68, in dumps
return _dumps(s, cls=cls or _default_encoder,
File "/usr/local/lib/python3.10/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/usr/local/lib/python3.10/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.10/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/local/lib/python3.10/site-packages/kombu/utils/json.py", line 58, in default
return super().default(o)
File "/usr/local/lib/python3.10/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
What I'm hearing is that this is caused by recent versions of Kombu using the python standard library's json module, instead of pypi's simplejson module ( https://docs.celeryq.dev/projects/kombu/en/stable/changelog.html#rc1 ).我听到的是,这是由使用 python 标准库的 json 模块而不是 pypi 的 simplejson 模块( https://docs.celeryq.dev/projects/kombu/en/stable/changelog.html #rc1 )。 Apparently the standard library's "json" module cannot (de)serialize bytes, but pypi's "simplejson" can.
显然,标准库的“json”模块不能(反)序列化字节,但 pypi 的“simplejson”可以。 Older versions of Celery+Kombu use simplejson by default.
旧版本的 Celery+Kombu 默认使用 simplejson。
EG:例如:
>>> import json
>>> s = b'abc'
>>> json.dumps(s)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/local/lib/python3.10/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.10/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/local/lib/python3.10/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
My team/project may or may not want to move to the json module's serialization eventually, but for now it seems to make the most sense to use simplejson for the sake of interoperability.我的团队/项目最终可能希望也可能不希望移动到 json 模块的序列化,但就目前而言,为了互操作性使用 simplejson 似乎是最有意义的。
(This is part of a largish CPython 2.7 -> CPython 3.10 port; I just want things to work on 3.10 first, and then worry about updating dependencies to be modern. Right now, the client and server are on different versions of celery and kombu) (这是较大的 CPython 2.7 -> CPython 3.10 端口的一部分;我只想首先在 3.10 上运行,然后担心更新依赖项以使其成为现代的。现在,客户端和服务器位于不同版本的 celery 和 kombu )
Thanks!谢谢!
Perhaps you can monkeypatch kombu from your code, after importing it, to use simplejson instead of json. From the traceback above:也许你可以从你的代码中使用 monkeypatch kombu,在导入它之后,使用 simplejson 而不是 json。从上面的回溯:
File "/usr/local/lib/python3.10/site-packages/kombu/serialization.py", line 210, in dumps
payload = encoder(data)
If encoder
is a global name, rebind kombu.encoder
to a function that uses simplejson.如果
encoder
是一个全局名称, kombu.encoder
重新绑定为使用 simplejson 的 function。 If not, but dumps
is, rebind kombu.dumps
.如果不是,但
dumps
是,重新绑定kombu.dumps
。
Or, this pair或者,这一对
File "/usr/local/lib/python3.10/site-packages/kombu/utils/json.py", line 68, in dumps
return _dumps(s, cls=cls or _default_encoder,
suggests that you can set kombu.utils.json.cls
to something other than None to change behavior.建议您可以将
kombu.utils.json.cls
设置为 None 以外的其他内容以更改行为。 Check the kombu docs or read the module code.检查 kombu 文档或阅读模块代码。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.