简体   繁体   English

是否可以将字符串或列表以外的数据作为 python 中的子进程的参数传递?

[英]Is it possible to pass data other than string or list as argument for a subprocess in python?

I can start a sub-process with a list as arguments for example like this:我可以使用 arguments 列表启动一个子进程,例如:

data = []
data.append("test")
subprocess.run(['python', 'myprocess.py'] + data)

My question is, if it is possible to pass bigger amounts of data to the sub-process?我的问题是,是否可以将大量数据传递给子流程? For example, if I have loaded a bunch of images and want to pass them directly as numpy arrays without having to store them to file and pass the paths as strings to load them again in the sub-process?例如,如果我加载了一堆图像并想直接将它们作为 numpy arrays 传递,而不必将它们存储到文件并将路径作为字符串传递以在子进程中再次加载它们?

Argument vectors are arrays of C strings at the operating system level (see man execve ).参数向量是操作系统级别的 C 字符串的 arrays(参见man execve )。 You can only pass data on the argv that can be represented as a C string (note that C strings are NUL-terminated, so no NULs allowed).您只能在 argv 上传递可以表示为 C 字符串的数据(请注意,C 字符串以 NUL 结尾,因此不允许使用 NUL)。 You can certainly serialize data into a C string on one side and deserialize it on the other side, but that's work.您当然可以在一侧将数据序列化为 C 字符串并在另一侧反序列化,但这是可行的。

Moreover, there's a maximum amount of storage available that's shared between environment variables and command-line arguments -- so the more/larger environment variables you have, the shorter the combined length of all your command-line arguments can be.此外,在环境变量和命令行 arguments 之间共享的最大可用存储量 - 因此您拥有的环境变量越多/越大,所有命令行 arguments 的组合长度就越短。

No you cannot.你不能。 The subprocess module is only a wrapper around the system call used to create a process and pass it a command line . subprocess模块只是系统调用的一个包装器,用于创建一个进程并传递给它一个命令行 In most system, that command line is just a sequence of space separated words, hence the list of strings.在大多数系统中,该命令行只是一系列空格分隔的单词,因此是字符串列表。

If you want to pass anything else, you will have to serialize the data on one side, pass it from one process to the other one through pipes or shared memory (if available) or environment variables and deserialize it in the second process.如果要传递其他任何内容,则必须在一侧序列化数据,通过管道或共享 memory(如果可用)或环境变量将其从一个进程传递到另一个进程,并在第二个进程中反序列化它。 pickle can be an easy tool to serialize simple data. pickle可以成为序列化简单数据的简单工具。

Alternatively, you could have a look at the multiprocess modules that does those things under the hood between Python processes或者,您可以查看在 Python 进程之间执行这些操作的multiprocess进程模块

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM