[英]python: read file continuously, even after it has been logrotated
I have a simple python script, where I read logfile continuosly (same as tail -f
) 我有一个简单的python脚本,我连续读取logfile(与
tail -f
相同)
while True:
line = f.readline()
if line:
print line,
else:
time.sleep(0.1)
How can I make sure that I can still read the logfile, after it has been rotated by logrotate? 在通过logrotate旋转后,如何确保仍然可以读取日志文件?
ie I need to do the same what tail -F
would do. 即我需要做同样的
tail -F
会做什么。
I am using python 2.7
我正在使用
python 2.7
As long as you only plan to do this on Unix, the most robust way is probably to check so that the open file still refers to the same i-node as the name, and reopen it when that is no longer the case. 只要您计划在Unix上执行此操作,最可靠的方法是检查以便打开的文件仍然引用与名称相同的i-node ,并在不再是这种情况时重新打开它。 You can get the i-number of the file from
os.stat
and os.fstat
, in the st_ino
field. 您可以在
st_ino
字段中从os.stat
和os.fstat
获取该文件的i-number。
It could look like this: 它可能看起来像这样:
import os, sys, time
name = "logfile"
current = open(name, "r")
curino = os.fstat(current.fileno()).st_ino
while True:
while True:
buf = current.read(1024)
if buf == "":
break
sys.stdout.write(buf)
try:
if os.stat(name).st_ino != curino:
new = open(name, "r")
current.close()
current = new
curino = os.fstat(current.fileno()).st_ino
continue
except IOError:
pass
time.sleep(1)
I doubt this works on Windows, but since you're speaking in terms of tail
, I'm guessing that's not a problem. 我怀疑这可以在Windows上运行,但是因为你说的是
tail
,我猜这不是问题。 :) :)
You can do it by keeping track of where you are in the file and reopening it when you want to read. 您可以通过跟踪文件中的位置并在想要阅读时重新打开来完成此操作。 When the log file rotates, you notice that the file is smaller and since you reopen, you handle any unlinking too.
当日志文件旋转时,您会注意到文件较小,并且在重新打开后,您也可以处理任何取消链接。
import time
cur = 0
while True:
try:
with open('myfile') as f:
f.seek(0,2)
if f.tell() < cur:
f.seek(0,0)
else:
f.seek(cur,0)
for line in f:
print line.strip()
cur = f.tell()
except IOError, e:
pass
time.sleep(1)
This example hides errors like file not found because I'm not sure of logrotate details such as small periods of time where the file is not available. 此示例隐藏了未找到文件之类的错误,因为我不确定logrotate详细信息,例如文件不可用的小时间段。
Thanks to @tdelaney and @Dolda2000's answers, I ended up with what follows. 感谢@tdelaney和@ Dolda2000的答案,我最终得到了以下内容。 It should work on both Linux and Windows, and also handle logrotate's
copytruncate
or create
options (respectively copy then truncate size to 0 and move then recreate file). 它应该适用于Linux和Windows,还可以处理logrotate的
copytruncate
或create
选项(分别复制然后截断大小为0然后移动然后重新创建文件)。
file_name = 'my_log_file'
seek_end = True
while True: # handle moved/truncated files by allowing to reopen
with open(file_name) as f:
if seek_end: # reopened files must not seek end
f.seek(0, 2)
while True: # line reading loop
line = f.readline()
if not line:
try:
if f.tell() > os.path.getsize(file_name):
# rotation occurred (copytruncate/create)
f.close()
seek_end = False
break
except FileNotFoundError:
# rotation occurred but new file still not created
pass # wait 1 second and retry
time.sleep(1)
do_stuff_with(line)
A limitation when using copytruncate
option is that if lines are appended to the file while time-sleeping, and rotation occurs before wake-up, the last lines will be "lost" (they will still be in the now "old" log file, but I cannot see a decent way to "follow" that file to finish reading it). 使用
copytruncate
选项时的一个限制是,如果在时间休眠时将行附加到文件,并且在唤醒之前发生旋转,则最后一行将“丢失”(它们仍将在现在的“旧”日志文件中,但我看不出一个体面的方式来“跟随”该文件来完成阅读它。 This limitation is not relevant with "move and create" create
option because f
descriptor will still point to the renamed file and therefore last lines will be read before the descriptor is closed and opened again. 此限制与“移动并创建”
create
选项无关,因为f
描述符仍将指向重命名的文件,因此在关闭描述符并再次打开之前将读取最后一行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.