[英]python subprocess Popen environment PATH?
I'm confused about how subprocess
searches for the executable when using Popen()
.我对使用
Popen()
时subprocess
Popen()
如何搜索可执行文件感到困惑。 It works if given absolute paths to the child process, but I'm trying to use relative paths.如果给定子进程的绝对路径,它就可以工作,但我正在尝试使用相对路径。 I've found that if I set the environment variable PYTHONPATH then I can get imported modules from that path ok, and PYTHONPATH is there in
sys.path
, but it doesn't seem to help with the behaviour of subprocess.Popen
.我发现如果我设置了环境变量 PYTHONPATH 那么我可以从那个路径中获取导入的模块,并且 PYTHONPATH 在
sys.path
,但它似乎对subprocess.Popen
的行为没有帮助。 I've also tried editing the sitecustomize.py
file adding PYTHONPATH to os.environ
, like so我还尝试编辑
sitecustomize.py
文件,将 PYTHONPATH 添加到os.environ
,就像这样
# copy PYTHONPATH environment variable into PATH to allow our stuff to use
# relative paths for subprocess spawning
import os
if os.getenv('PYTHONPATH') is not None and os.getenv('PATH') is not none:
os.environ['PATH'] = ':'.join([os.getenv('PATH'), os.getenv('PYTHONPATH')])
and verified that when starting up python , either interactively, with ipython, or by running a script from the command line, that PYTHONPATH is successfully appearing in os.environ
.并验证在启动 python 时,无论是交互方式,使用 ipython,还是通过从命令行运行脚本,PYTHONPATH 成功出现在
os.environ
。 However, subrocess.Popen
still doesn't search there for the executable.但是,
subrocess.Popen
仍然不会在那里搜索可执行文件。 I thought it was supposed to inherit the parents environment, if no env
kwarg is specified?如果没有指定
env
kwarg,我认为它应该继承父环境? Next I tried giving env
explicitly, first by making a copy of os.getenv
and secondly just by giving env={'PATH': '/explicit/path/to/search/from'}
, and it still does not find the executable.接下来我尝试明确地给出
env
,首先通过制作os.getenv
的副本,其次只是通过给出env={'PATH': '/explicit/path/to/search/from'}
,它仍然没有找到可执行文件. Now I'm stumped.现在我很难过。
Hopefully an example will help explain my problem more clearly:希望一个例子将有助于更清楚地解释我的问题:
/dir/subdir1/some_executable /dir/subdir1/some_executable
/dir/subdir2/some_script.py /dir/subdir2/some_script.py
# some_script.py
from subprocess import Popen, PIPE
spam, eggs = Popen(['../subdir1/some_executable'], stdout=PIPE, stderr=PIPE).communicate()
If I'm in /dir/subdir2
and I run python some_script.py
it works, but if I'm in /dir
and I run python subdir2/some_script.py
even though /dir/subdir2
is in the os.environ['PATH']
, then subprocess will throw OSError: [Errno 2] No such file or directory
.如果我在
/dir/subdir2
并且我运行python some_script.py
它可以工作,但是如果我在/dir
并且我运行python subdir2/some_script.py
即使/dir/subdir2
在os.environ['PATH']
,然后子进程将抛出OSError: [Errno 2] No such file or directory
。
(filling in details from a comment to make a separate answer) (从评论中填写详细信息以单独回答)
First off, relative paths (paths containing slashes) never get checked in any PATH
, no matter what you do.首先,无论您做什么,都不会在任何
PATH
检查相对路径(包含斜杠的路径)。 They are relative to the current working directory only.它们仅相对于当前工作目录。 If you need to resolve relative paths, you will have to search the
PATH
manually, or munge the PATH
to include the subdirectories and then just use the command name as in my suggestion below.如果您需要解析相对路径,则必须手动搜索
PATH
,或者修改PATH
以包含子目录,然后只需使用我下面的建议中的命令名称。
If you want to run a program relative to the location of the Python script , use __file__
and go from there to find the absolute path of the program, and then use the absolute path in Popen
.如果要运行相对于 Python 脚本位置的程序,请使用
__file__
并从那里找到程序的绝对路径,然后使用Popen
的绝对路径。
PATH
PATH
Secondly, there is an issue in the Python bug tracker about how Python deals with bare commands (no slashes).其次, Python 错误跟踪器中存在关于 Python 如何处理裸命令(无斜杠)的问题。 Basically, on Unix/Mac
Popen
behaves like os.execvp
when the argument env=None
(some unexpected behavior has been observed and noted at the end):基本上,在Unix / Mac的
Popen
行为就像os.execvp
当参数env=None
(一些意想不到的行为已经观察到,并指出在最后):
On POSIX, the class uses
os.execvp()
-like behavior to execute the child program.在 POSIX 上,该类使用
os.execvp()
类的行为来执行子程序。
This is actually true for both shell=False
and shell=True
, provided env=None
.这对于
shell=False
和shell=True
实际上都是shell=True
,前提是env=None
。 What this behavior means is explained in the documentation of the function os.execvp
:函数
os.execvp
的文档中解释了此行为的含义:
The variants which include a “p” near the end (
execlp()
,execlpe()
,execvp()
, andexecvpe()
) will use thePATH
environment variable to locate the program file .在末尾包含“p”的变体(
execlp()
、execlpe()
、execvp()
和execvpe()
)将使用PATH
环境变量来定位程序文件。 When the environment is being replaced (using one of theexec*e
variants, discussed in the next paragraph), the new environment is used as the source of thePATH
variable.当环境被替换时(使用
exec*e
变体之一,在下一段中讨论),新环境被用作PATH
变量的源。
For
execle()
,execlpe()
,execve()
, andexecvpe()
(note that these all end in “e”), the env parameter must be a mapping which is used to define the environment variables for the new process (these are used instead of the current process' environment);对于
execlpe()
execle()
、execlpe()
、execve()
和execvpe()
(注意这些都以“e”结尾), env参数必须是一个映射,用于定义新进程的环境变量(这些用于代替当前进程的环境); the functionsexecl()
,execlp()
,execv()
, andexecvp()
all cause the new process to inherit the environment of the current process.函数
execl()
、execlp()
、execv()
和execvp()
都会导致新进程继承当前进程的环境。
The second quoted paragraph implies that execvp
will use the current process' environment variables.引用的第二段暗示
execvp
将使用当前进程的环境变量。 Combined with the first quoted paragraph, we deduce that execvp
will use the value of the environment variable PATH
from the environment of the current process.结合第一个引用的段落,我们推断
execvp
会从当前进程的环境中使用环境变量PATH
的值。 This means that Popen
looks at the value of PATH
as it was when Python launched (the Python that runs the Popen
instantiation) and no amount of changing os.environ
will help you fix that.这意味着
Popen
查看PATH
的值,因为它在 Python 启动时(运行Popen
实例化的 Python)并且没有任何更改os.environ
可以帮助您解决这个问题。
Also, on Windows with shell=False
, Popen
pays no attention to PATH
at all, and will only look in relative to the current working directory.此外,在带有
shell=False
Windows 上, Popen
根本不关注PATH
,只会查看相对于当前工作目录的内容。
shell=True
does shell=True
做什么What happens if we pass shell=True
to Popen
?如果我们将
shell=True
传递给Popen
会发生什么? In that case, Popen
simply calls the shell :在这种情况下,
Popen
只需调用 shell :
The shell argument (which defaults to
False
) specifies whether to use the shell as the program to execute.shell参数(默认为
False
)指定是否使用 shell 作为要执行的程序。
That is to say,
Popen
does the equivalent of:也就是说,
Popen
作用相当于:Popen(['/bin/sh', '-c', args[0], args[1], ...])
In other words, with shell=True
Python will directly execute /bin/sh
, without any searching (passing the argument executable
to Popen
can change this, and it seems that if it is a string without slashes, then it will be interpreted by Python as the shell program's name to search for in the value of PATH
from the environment of the current process, ie, as it searches for programs in the case shell=False
described above).换句话说,with
shell=True
Python 会直接执行/bin/sh
,无需任何搜索(将参数executable
传递给Popen
可以改变这一点,而且看起来如果它是一个没有斜线的字符串,那么它会被 Python 解释作为 shell 程序的名称,在当前进程的环境中在PATH
的值中搜索,即,当它在上述shell=False
的情况下搜索程序时)。
In turn, /bin/sh
(or our shell executable
) will look for the program we want to run in its own environment's PATH
, which is the same as the PATH
of the Python (current process), as deduced from the code after the phrase "That is to say..." above (because that call has shell=False
, so it is the case already discussed earlier).反过来,
/bin/sh
(或我们的 shell executable
)将在其自身环境的PATH
查找我们想要运行的程序,该程序与 Python(当前进程)的PATH
相同,从后面的代码推断上面的短语“也就是说...”(因为该调用具有shell=False
,因此前面已经讨论过这种情况)。 Therefore, the execvp
-like behavior is what we get with both shell=True
and shell=False
, as long as env=None
.因此,只要
env=None
,我们就可以通过shell=True
和shell=False
类似execvp
的行为。
env
to Popen
env
传递给Popen
So what happens if we pass env=dict(PATH=...)
to Popen
(thus defining an environment variable PATH
in the environment of the program that will be run by Popen
)?那么如果我们将
env=dict(PATH=...)
传递给Popen
(从而在由Popen
运行的程序的环境中定义一个环境变量PATH
)会发生什么?
In this case, the new environment is used to search for the program to execute.在这种情况下,新环境用于搜索要执行的程序。 Quoting the documentation of
Popen
:引用的文件
Popen
:
If env is not
None
, it must be a mapping that defines the environment variables for the new process;如果env不是
None
,则它必须是定义新进程的环境变量的映射; these are used instead of the default behavior of inheriting the current process' environment.这些用于代替继承当前进程环境的默认行为。
Combined with the above observations, and from experiments using Popen
, this means that Popen
in this case behaves like the function os.execvpe
.与上述观察相结合,并从实验使用
Popen
,这意味着Popen
在这种情况下表现得如同功能os.execvpe
。 If shell=False
, Python searches for the given program in the newly defined PATH
.如果
shell=False
,Python 在新定义的PATH
搜索给定的程序。 As already discussed above for shell=True
, in that case the program is either /bin/sh
, or, if a program name is given with the argument executable
, then this alternative (shell) program is searched for in the newly defined PATH
.正如上面针对
shell=True
已经讨论的那样,在这种情况下,程序是/bin/sh
,或者,如果程序名称与参数executable
一起给出,则在新定义的PATH
搜索此替代(shell)程序。
In addition, if shell=True
, then inside the shell the search path that the shell will use to find the program given in args
is the value of PATH
passed to Popen
via env
.此外,如果
shell=True
,则在 shell 内,shell用于查找args
给出的程序的搜索路径是PATH
通过env
传递给Popen
的值。
So with env != None
, Popen
searches in the value of the key PATH
of env
(if a key PATH
is present in env
).因此,与
env != None
, Popen
搜索在键值PATH
的env
(如果关键PATH
出现在env
)。
PATH
as argumentsPATH
以外的环境变量作为参数传播There is a caveat about environment variables other than PATH
: if the values of those variables are needed in the command (eg, as command-line arguments to the program being run), then even if these are present in the env
given to Popen
, they will not get interpreted without shell=True
.关于
PATH
以外的环境变量有一个警告:如果命令中需要这些变量的值(例如,作为正在运行的程序的命令行参数),那么即使这些变量存在于提供给Popen
的env
,如果没有shell=True
它们将不会被解释。 This is easily avoided without changing shell=True
: insert those value directly in the list
argument args
that is given to Popen
.这很容易避免,而无需更改
shell=True
:将这些值直接插入提供给Popen
的list
参数args
中。 (Also, if these values come from Python's own environment, the method os.environ.get
can be used to get their values). (此外,如果这些值来自 Python 自身的环境,则可以使用
os.environ.get
方法来获取它们的值)。
/usr/bin/env
/usr/bin/env
If you JUST need path evaluation and don't really want to run your command line through a shell, and are on UNIX, I advise using env
instead of shell=True
, as in如果您只需要路径评估并且不想通过 shell 运行命令行,并且在 UNIX 上,我建议使用
env
而不是shell=True
,如
path = '/dir1:/dir2'
subprocess.Popen(['/usr/bin/env', '-P', path, 'progtorun', other, args], ...)
This lets you pass a different PATH
to the env
process (using the option -P
), which will use it to find the program.这使您可以将不同的
PATH
传递给env
进程(使用选项-P
),它将使用它来查找程序。 It also avoids issues with shell metacharacters and potential security issues with passing arguments through the shell.它还避免了 shell 元字符问题和通过 shell 传递参数的潜在安全问题。 Obviously, on Windows (pretty much the only platform without a
/usr/bin/env
) you will need to do something different.显然,在 Windows(几乎是唯一没有
/usr/bin/env
)上,您需要做一些不同的事情。
shell=True
shell=True
Quoting the Popen
documentation:引用
Popen
文档:
If shell is
True
, it is recommended to passargs
as a string rather than as a sequence.如果shell是
True
,建议将args
作为字符串而不是序列传递。
Note: Read the Security Considerations section before using
shell=True
.注意:在使用
shell=True
之前阅读安全注意事项部分。
The following behavior was observed:观察到以下行为:
This call raises FileNotFoundError
, as expected:正如预期的那样,此调用引发
FileNotFoundError
:
subprocess.call(['sh'], shell=False, env=dict(PATH=''))
This call finds sh
, which is unexpected:这个调用找到了
sh
,这是出乎意料的:
subprocess.call(['sh'], shell=False, env=dict(FOO=''))
Typing echo $PATH
inside the shell that this opens reveals that the PATH
value is not empty, and also different from the value of PATH
in the environment of Python.打字
echo $PATH
这将打开外壳里面揭示了PATH
值不为空,并从价值也不同PATH
在Python的环境。 So it seems that PATH
was indeed not inherited from Python (as expected in the presence of env != None
), but still, it the PATH
is nonempty.因此,似乎
PATH
确实不是从 Python 继承的(正如在env != None
存在的情况下所预期的那样),但是,它PATH
仍然是非空的。 Unknown why this is the case.不知道为什么会这样。
This call raises FileNotFoundError
, as expected:正如预期的那样,此调用引发
FileNotFoundError
:
subprocess.call(['tree'], shell=False, env=dict(FOO=''))
This finds tree
, as expected:这会发现
tree
,正如预期的那样:
subprocess.call(['tree'], shell=False, env=None)
You appear to be a little confused about the nature of PATH
and PYTHONPATH
.您似乎对
PATH
和PYTHONPATH
的性质有些困惑。
PATH
is an environment variable that tells the OS shell where to search for executables. PATH
是一个环境变量,它告诉 OS shell 在哪里搜索可执行文件。
PYTHONPATH
is an environment variable that tells the Python interpreter where to search for modules to import. PYTHONPATH
是一个环境变量,它告诉 Python 解释器在哪里搜索要导入的模块。 It has nothing to do with subprocess
finding executable files.它与
subprocess
进程查找可执行文件无关。
Due to the differences in the underlying implementation, subprocess.Popen
will only search the path by default on non-Windows systems (Windows has some system directories it always searches, but that's distinct from PATH
processing).由于底层实现的差异,
subprocess.Popen
只会在非 Windows 系统上默认搜索路径(Windows 有一些它总是搜索的系统目录,但这与PATH
处理不同)。 The only reliable cross-platform way to scan the path is by passing shell=True
to the subprocess call, but that has its own issues (as detailed in the Popen
documentation )扫描路径的唯一可靠的跨平台方法是将
shell=True
传递给子进程调用,但这有其自身的问题(如Popen
文档中所述)
However, it appears your main problem is that you are passing a path fragment to Popen
rather than a simple file name.但是,您的主要问题似乎是您将路径片段传递给
Popen
而不是简单的文件名。 As soon as you have a directory separator in there, you're going to disable the PATH
search, even on a non-Windows platform (eg see the Linux documentation for the exec family of functions ).只要您在那里有了目录分隔符,您就会禁用
PATH
搜索,即使在非 Windows 平台上也是如此(例如,请参阅exec 函数系列的 Linux 文档)。
A relative path in subprocess.Popen acts relative to the current working directory, not the elements of the systems PATH. subprocess.Popen 中的相对路径相对于当前工作目录起作用,而不是系统路径的元素。 If you run
python subdir2/some_script.py
from /dir
then the expected executable location (passed to Popen) will be /dir/../subdir1/some_executable
, aka /subdir1/some_executable
not /dir/subdir1/some_executable
.如果您从
/dir
运行python subdir2/some_script.py
那么预期的可执行位置(传递给/dir/../subdir1/some_executable
)将是/dir/../subdir1/some_executable
,又名/subdir1/some_executable
而不是/dir/subdir1/some_executable
。
If you would definitely like to use relative paths from a scripts own directory to a particular executable the best option would be to first construct an absolute path from the directory portion of the __file__
global variable.如果您肯定想使用从脚本自己的目录到特定可执行文件的相对路径,最好的选择是首先从
__file__
全局变量的目录部分构造一个绝对路径。
#/usr/bin/env python
from subprocess import Popen, PIPE
from os.path import abspath, dirname, join
path = abspath(join(dirname(__file__), '../subdir1/some_executable'))
spam, eggs = Popen(path, stdout=PIPE, stderr=PIPE).communicate()
The pythonpath is set to the path from where the python interpreter is executed. pythonpath 设置为执行 python 解释器的路径。 So, in second case of your example, the path is set to /dir and not /dir/subdir2 That's why you get an error.
因此,在您的示例的第二种情况下,路径设置为 /dir 而不是 /dir/subdir2 这就是您收到错误的原因。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.