[英]Redirect stdout to a file in Python?
How do I redirect stdout to an arbitrary file in Python?如何将标准输出重定向到 Python 中的任意文件?
When a long-running Python script (eg, web application) is started from within the ssh session and backgounded, and the ssh session is closed, the application will raise IOError and fail the moment it tries to write to stdout. When a long-running Python script (eg, web application) is started from within the ssh session and backgounded, and the ssh session is closed, the application will raise IOError and fail the moment it tries to write to stdout. I needed to find a way to make the application and modules output to a file rather than stdout to prevent failure due to IOError.
我需要找到一种方法将应用程序和模块 output 制作成文件而不是标准输出,以防止由于 IOError 而导致失败。 Currently, I employ nohup to redirect output to a file, and that gets the job done, but I was wondering if there was a way to do it without using nohup, out of curiosity.
目前,我使用 nohup 将 output 重定向到一个文件,这样就完成了工作,但出于好奇,我想知道是否有办法不使用 nohup 来做到这一点。
I have already tried sys.stdout = open('somefile', 'w')
, but this does not seem to prevent some external modules from still outputting to terminal (or maybe the sys.stdout =...
line did not fire at all).我已经尝试过
sys.stdout = open('somefile', 'w')
,但这似乎并不能阻止某些外部模块仍然输出到终端(或者sys.stdout =...
行没有触发全部)。 I know it should work from simpler scripts I've tested on, but I also didn't have time yet to test on a web application yet.我知道它应该可以通过我测试过的更简单的脚本来工作,但我还没有时间在 web 应用程序上进行测试。
If you want to do the redirection within the Python script, setting sys.stdout
to a file object does the trick:如果您想在 Python 脚本中进行重定向,请将
sys.stdout
设置为文件对象即可:
import sys
sys.stdout = open('file', 'w')
print('test')
sys.stdout.close()
A far more common method is to use shell redirection when executing (same on Windows and Linux):一种更常见的方法是在执行时使用 shell 重定向(在 Windows 和 Linux 上相同):
$ python foo.py > file
There is contextlib.redirect_stdout()
function in Python 3.4+: Python 3.4+ 中有 contextlib.redirect_stdout
contextlib.redirect_stdout()
函数:
from contextlib import redirect_stdout
with open('help.txt', 'w') as f:
with redirect_stdout(f):
print('it now prints to `help.text`')
It is similar to:它类似于:
import sys
from contextlib import contextmanager
@contextmanager
def redirect_stdout(new_target):
old_target, sys.stdout = sys.stdout, new_target # replace sys.stdout
try:
yield new_target # run some code with the replaced stdout
finally:
sys.stdout = old_target # restore to the previous value
that can be used on earlier Python versions.可以在早期的 Python 版本上使用。 The latter version is not reusable .
后一个版本不可 重复使用。 It can be made one if desired.
如果需要,它可以制作一个。
It doesn't redirect the stdout at the file descriptors level eg:它不会在文件描述符级别重定向标准输出,例如:
import os
from contextlib import redirect_stdout
stdout_fd = sys.stdout.fileno()
with open('output.txt', 'w') as f, redirect_stdout(f):
print('redirected to a file')
os.write(stdout_fd, b'not redirected')
os.system('echo this also is not redirected')
b'not redirected'
and 'echo this also is not redirected'
are not redirected to the output.txt
file. b'not redirected'
和'echo this also is not redirected'
不会重定向到output.txt
文件。
To redirect at the file descriptor level, os.dup2()
could be used:要在文件描述符级别重定向,可以使用
os.dup2()
:
import os
import sys
from contextlib import contextmanager
def fileno(file_or_fd):
fd = getattr(file_or_fd, 'fileno', lambda: file_or_fd)()
if not isinstance(fd, int):
raise ValueError("Expected a file (`.fileno()`) or a file descriptor")
return fd
@contextmanager
def stdout_redirected(to=os.devnull, stdout=None):
if stdout is None:
stdout = sys.stdout
stdout_fd = fileno(stdout)
# copy stdout_fd before it is overwritten
#NOTE: `copied` is inheritable on Windows when duplicating a standard stream
with os.fdopen(os.dup(stdout_fd), 'wb') as copied:
stdout.flush() # flush library buffers that dup2 knows nothing about
try:
os.dup2(fileno(to), stdout_fd) # $ exec >&to
except ValueError: # filename
with open(to, 'wb') as to_file:
os.dup2(to_file.fileno(), stdout_fd) # $ exec > to
try:
yield stdout # allow code to be run with the redirected stdout
finally:
# restore stdout to its previous value
#NOTE: dup2 makes stdout_fd inheritable unconditionally
stdout.flush()
os.dup2(copied.fileno(), stdout_fd) # $ exec >&copied
The same example works now if stdout_redirected()
is used instead of redirect_stdout()
:如果使用
stdout_redirected()
而不是redirect_stdout()
则相同的示例现在可以工作:
import os
import sys
stdout_fd = sys.stdout.fileno()
with open('output.txt', 'w') as f, stdout_redirected(f):
print('redirected to a file')
os.write(stdout_fd, b'it is redirected now\n')
os.system('echo this is also redirected')
print('this is goes back to stdout')
The output that previously was printed on stdout now goes to output.txt
as long as stdout_redirected()
context manager is active.只要
stdout_redirected()
上下文管理器处于活动状态,以前打印在 stdout 上的输出现在就会转到output.txt
。
Note: stdout.flush()
does not flush C stdio buffers on Python 3 where I/O is implemented directly on read()
/ write()
system calls.注意:
stdout.flush()
不会在 Python 3 上刷新 C stdio 缓冲区,其中 I/O 直接在read()
/ write()
系统调用上实现。 To flush all open C stdio output streams, you could call libc.fflush(None)
explicitly if some C extension uses stdio-based I/O:要刷新所有打开的 C stdio 输出流,如果某些 C 扩展使用基于 stdio 的 I/O,您可以显式调用
libc.fflush(None)
:
try:
import ctypes
from ctypes.util import find_library
except ImportError:
libc = None
else:
try:
libc = ctypes.cdll.msvcrt # Windows
except OSError:
libc = ctypes.cdll.LoadLibrary(find_library('c'))
def flush(stream):
try:
libc.fflush(None)
stream.flush()
except (AttributeError, ValueError, IOError):
pass # unsupported
You could use stdout
parameter to redirect other streams, not only sys.stdout
eg, to merge sys.stderr
and sys.stdout
:您可以使用
stdout
参数来重定向其他流,而不仅仅是sys.stdout
例如,以合并sys.stderr
和sys.stdout
:
def merged_stderr_stdout(): # $ exec 2>&1
return stdout_redirected(to=sys.stdout, stdout=sys.stderr)
Example:例子:
from __future__ import print_function
import sys
with merged_stderr_stdout():
print('this is printed on stdout')
print('this is also printed on stdout', file=sys.stderr)
Note: stdout_redirected()
mixes buffered I/O ( sys.stdout
usually) and unbuffered I/O (operations on file descriptors directly).注意:
stdout_redirected()
混合了缓冲 I/O(通常是sys.stdout
)和非缓冲 I/O(直接操作文件描述符)。 Beware, there could be buffering issues .当心,可能存在缓冲问题。
To answer, your edit: you could use python-daemon
to daemonize your script and use logging
module (as @erikb85 suggested ) instead of print
statements and merely redirecting stdout for your long-running Python script that you run using nohup
now.要回答,您的编辑:您可以使用
python-daemon
来守护您的脚本并使用logging
模块(如@erikb85 建议的那样)而不是print
语句,并且仅为您现在使用nohup
运行的长时间运行的 Python 脚本重定向标准输出。
you can try this too much better你可以试试这个更好
import sys
class Logger(object):
def __init__(self, filename="Default.log"):
self.terminal = sys.stdout
self.log = open(filename, "a")
def write(self, message):
self.terminal.write(message)
self.log.write(message)
sys.stdout = Logger("yourlogfilename.txt")
print "Hello world !" # this is should be saved in yourlogfilename.txt
The other answers didn't cover the case where you want forked processes to share your new stdout.其他答案没有涵盖您希望分叉进程共享新标准输出的情况。
To do that:要做到这一点:
from os import open, close, dup, O_WRONLY
old = dup(1)
close(1)
open("file", O_WRONLY) # should open on 1
..... do stuff and then restore
close(1)
dup(old) # should dup to 1
close(old) # get rid of left overs
Quoted from PEP 343 -- The "with" Statement (added import statement):引自PEP 343——“with”语句(添加导入语句):
Redirect stdout temporarily:暂时重定向标准输出:
import sys
from contextlib import contextmanager
@contextmanager
def stdout_redirected(new_stdout):
save_stdout = sys.stdout
sys.stdout = new_stdout
try:
yield None
finally:
sys.stdout = save_stdout
Used as follows:用法如下:
with open(filename, "w") as f:
with stdout_redirected(f):
print "Hello world"
This isn't thread-safe, of course, but neither is doing this same dance manually.当然,这不是线程安全的,但也不是手动执行相同的舞蹈。 In single-threaded programs (for example in scripts) it is a popular way of doing things.
在单线程程序中(例如在脚本中),这是一种流行的做事方式。
import sys
sys.stdout = open('stdout.txt', 'w')
Here is a variation of Yuda Prawira answer:这是Yuda Prawira答案的变体:
flush()
and all the file attributesflush()
和所有文件属性stderr
alsostderr
. .
import contextlib, sys
@contextlib.contextmanager
def log_print(file):
# capture all outputs to a log file while still printing it
class Logger:
def __init__(self, file):
self.terminal = sys.stdout
self.log = file
def write(self, message):
self.terminal.write(message)
self.log.write(message)
def __getattr__(self, attr):
return getattr(self.terminal, attr)
logger = Logger(file)
_stdout = sys.stdout
_stderr = sys.stderr
sys.stdout = logger
sys.stderr = logger
try:
yield logger.log
finally:
sys.stdout = _stdout
sys.stderr = _stderr
with log_print(open('mylogfile.log', 'w')):
print('hello world')
print('hello world on stderr', file=sys.stderr)
# you can capture the output to a string with:
# with log_print(io.StringIO()) as log:
# ....
# print('[captured output]', log.getvalue())
You need a terminal multiplexer like either tmux or GNU screen您需要一个终端多路复用器,例如tmux或GNU screen
I'm surprised that a small comment by Ryan Amos' to the original question is the only mention of a solution far preferable to all the others on offer, no matter how clever the python trickery may be and how many upvotes they've received.我很惊讶瑞安·阿莫斯 (Ryan Amos) 对原始问题的一个小评论是唯一提到的解决方案比提供的所有其他解决方案更可取,无论 python 技巧可能多么聪明以及他们收到了多少赞成票。 Further to Ryan's comment, tmux is a nice alternative to GNU screen.
除了 Ryan 的评论之外,tmux 是 GNU screen 的一个很好的替代品。
But the principle is the same: if you ever find yourself wanting to leave a terminal job running while you log-out, head to the cafe for a sandwich, pop to the bathroom, go home (etc) and then later, reconnect to your terminal session from anywhere or any computer as though you'd never been away, terminal multiplexers are the answer.但原则是一样的:如果您发现自己想在注销时让终端工作继续运行,请前往咖啡馆吃三明治,去洗手间,回家(等),然后重新连接到您的从任何地方或任何计算机终端会话,就好像你从来没有离开,终端复用器是答案。 Think of them as VNC or remote desktop for terminal sessions.
将它们视为终端会话的 VNC 或远程桌面。 Anything else is a workaround.
其他任何方法都是解决方法。 As a bonus, when the boss and/or partner comes in and you inadvertently ctrl-w / cmd-w your terminal window instead of your browser window with its dodgy content, you won't have lost the last 18 hours-worth of processing!
作为奖励,当老板和/或合作伙伴进来并且您无意中 ctrl-w / cmd-w 您的终端窗口而不是带有可疑内容的浏览器窗口时,您将不会丢失过去 18 小时的处理时间!
Based on this answer: https://stackoverflow.com/a/5916874/1060344 , here is another way I figured out which I use in one of my projects.基于这个答案: https : //stackoverflow.com/a/5916874/1060344 ,这是我在我的一个项目中使用的另一种方法。 For whatever you replace
sys.stderr
or sys.stdout
with, you have to make sure that the replacement complies with file
interface, especially if this is something you are doing because stderr/stdout are used in some other library that is not under your control.无论您用什么替换
sys.stderr
或sys.stdout
,您都必须确保替换符合file
接口,特别是如果这是您正在做的事情,因为 stderr/stdout 用于其他不受您控制的库中. That library may be using other methods of file object.该库可能正在使用文件对象的其他方法。
Check out this way where I still let everything go do stderr/stdout (or any file for that matter) and also send the message to a log file using Python's logging facility (but you can really do anything with this):看看这种方式,我仍然让一切都去执行 stderr/stdout(或任何与此相关的文件),并使用 Python 的日志记录工具将消息发送到日志文件(但你真的可以用它做任何事情):
class FileToLogInterface(file):
'''
Interface to make sure that everytime anything is written to stderr, it is
also forwarded to a file.
'''
def __init__(self, *args, **kwargs):
if 'cfg' not in kwargs:
raise TypeError('argument cfg is required.')
else:
if not isinstance(kwargs['cfg'], config.Config):
raise TypeError(
'argument cfg should be a valid '
'PostSegmentation configuration object i.e. '
'postsegmentation.config.Config')
self._cfg = kwargs['cfg']
kwargs.pop('cfg')
self._logger = logging.getlogger('access_log')
super(FileToLogInterface, self).__init__(*args, **kwargs)
def write(self, msg):
super(FileToLogInterface, self).write(msg)
self._logger.info(msg)
Programs written in other languages (eg C) have to do special magic (called double-forking) expressly to detach from the terminal (and to prevent zombie processes).用其他语言(例如 C)编写的程序必须使用特殊的魔法(称为双分叉)来明确地与终端分离(并防止僵尸进程)。 So, I think the best solution is to emulate them.
所以,我认为最好的解决方案是模仿它们。
A plus of re-executing your program is, you can choose redirections on the command-line, eg /usr/bin/python mycoolscript.py 2>&1 1>/dev/null
重新执行程序的一个
/usr/bin/python mycoolscript.py 2>&1 1>/dev/null
是,您可以在命令行上选择重定向,例如/usr/bin/python mycoolscript.py 2>&1 1>/dev/null
See this post for more info: What is the reason for performing a double fork when creating a daemon?有关更多信息,请参阅此帖子: 创建守护进程时执行双叉的原因是什么?
I know this question is answered (using python abc.py > output.log 2>&1
), but I still have to say:我知道这个问题得到了回答(使用
python abc.py > output.log 2>&1
),但我还是要说:
When writing your program, don't write to stdout.编写程序时,不要写入标准输出。 Always use logging to output whatever you want.
始终使用日志记录输出您想要的任何内容。 That would give you a lot of freedom in the future when you want to redirect, filter, rotate the output files.
将来当您想要重定向、过滤、旋转输出文件时,这会给您很大的自由。
As mentioned by @jfs, most solutions will not properly handle some types of stdout output such as that from C extensions.正如@jfs 所述,大多数解决方案将无法正确处理某些类型的标准输出 output,例如来自 C 扩展的标准输出。 There is a module that takes care of all this on PyPI called
wurlitzer
. PyPI 上有一个名为
wurlitzer
的模块可以处理所有这些问题。 You just need its sys_pipes
context manager.您只需要它的
sys_pipes
上下文管理器。 It's as easy as using:它就像使用一样简单:
from contextlib import redirect_stdout
import os
from wurlitzer import sys_pipes
log = open("test.log", "a")
with redirect_stdout(log), sys_pipes():
print("print statement")
os.system("echo echo call")
Based on previous answers on this post I wrote this class for myself as a more compact and flexible way of redirecting the output of pieces of code - here just to a list - and ensure that the output is normalized afterwards.根据之前对这篇文章的回答,我为自己编写了这个 class 作为一种更紧凑和更灵活的方式来重定向代码段 output - 这里只是一个列表 - 并确保 Z78E6221F6393D1356681DB398CED4 之后是正常的。
class out_to_lt():
def __init__(self, lt):
if type(lt) == list:
self.lt = lt
else:
raise Exception("Need to pass a list")
def __enter__(self):
import sys
self._sys = sys
self._stdout = sys.stdout
sys.stdout = self
return self
def write(self,txt):
self.lt.append(txt)
def __exit__(self, type, value, traceback):
self._sys.stdout = self._stdout
Used as:用作:
lt = []
with out_to_lt(lt) as o:
print("Test 123\n\n")
print(help(str))
Updating.更新。 Just found a scenario where I had to add two extra methods, but was easy to adapt:
刚刚发现一个场景,我必须添加两个额外的方法,但很容易适应:
class out_to_lt():
...
def isatty(self):
return True #True: You're running in a real terminal, False:You're being piped, redirected, cron
def flush(self):
pass
There are other versions using context but nothing this simple.还有其他版本使用上下文,但没有这么简单。 I actually just googled to double check it would work and was surprised not to see it, so for other people looking for a quick solution that is safe and directed at only the code within the context block, here it is:
实际上,我只是用谷歌搜索以仔细检查它是否可以工作,但很惊讶没有看到它,因此对于寻找安全且仅针对上下文块中的代码的快速解决方案的其他人来说,这里是:
import sys
with open('test_file', 'w') as sys.stdout:
print('Testing 1 2 3')
Tested like so:像这样测试:
$ cat redirect_stdout.py import sys with open('test_file', 'w') as sys.stdout: print('Testing 1 2 3') $ python redirect_stdout.py $ cat test_file Testing 1 2 3
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.