[英]Bash Terminal: Write only specific lines to logfile
I'm running simulation with lots of terminal output, which would exceed my disc space, if I'd save it to a logfile (eg by "cmd > logfile").我正在使用大量终端 output 运行模拟,如果我将其保存到日志文件(例如通过“cmd > logfile”),这将超出我的磁盘空间。 Now I would like to follow the entire terminal output, but at the same time I would like to save specific data/lines/values from this output to a file.
现在我想关注整个终端 output,但同时我想将这个 output 中的特定数据/行/值保存到文件中。
1) Is there a way to that in bash? 1)在bash中有没有办法做到这一点?
2) Or as alternative: Is it possible to save the logfile, extract the needed data and then delete the processed lines to avoid generating a huge logfile? 2)或者作为替代方案:是否可以保存日志文件,提取所需的数据,然后删除已处理的行以避免生成巨大的日志文件?
If you want to save into logfile
only the output containing mypattern
but you want the see all the output at the terminal, you could issue:如果您只想将包含
mypattern
的 output 保存到logfile
中,但您希望在终端查看所有 output,您可以发出:
cmd 2>&1 | tee /dev/tty | grep 'mypattern' > logfile
I also assumed that the output of cmd
may be directed to the standard output stream as well as to the standard error stream, by adding 2>&1
after cmd
. I also assumed that the output of
cmd
may be directed to the standard output stream as well as to the standard error stream, by adding 2>&1
after cmd
.
What criteria are you using to decide which lines to keep?您使用什么标准来决定保留哪些行?
One common filter is to just store stderr.一种常见的过滤器是只存储标准错误。
cmdlist 2>logfile # stdout still to console
For a more sophisticated filter, if you have specific patterns you want to save to the log, you can use sed
.对于更复杂的过滤器,如果您想要将特定模式保存到日志中,您可以使用
sed
。 Here's a simplistic example -这是一个简单的例子 -
seq 1 100 | sed '/.*[37]$/w tmplog'
This will generate numbers from 1 to 100 and send them all to the console, but capture all numbers that end with 3
or 7
to tmplog
.这将生成从 1 到 100 的数字并将它们全部发送到控制台,但会将所有以
3
或7
结尾的数字捕获到tmplog
。 It can also accept more complex lists of commands to help you be more comprehensive -它还可以接受更复杂的命令列表,以帮助您更全面 -
seq 1 100 | sed '/.*[37]$/w 37.log
/^2/w 37.log'
c.f. c.f. the
sed
manual for more detailed breakdowns. sed
手册了解更详细的故障。
You probably also want error output, so it might be a good idea to save that too.您可能还需要错误 output,因此最好也保存它。
seq 1 100 2>errlog | sed '/.*[37]$/w patlog'
For a more complex space-saving plan, create a named pipe, and compress the log from that in a background process.对于更复杂的节省空间计划,创建一个名为 pipe 的文件,并在后台进程中压缩日志。
$: mkfifo transfer # creates a queue file on disk that
$: gzip < transfer > log.gz & # reads from FIFO in bg, compresses to log
$: seq 1 100 2>&1 | tee transfer # tee writes one copy to stdout, one to file
This will show all the output as it comes, but also duplicate a copy to the named pipe;这将显示所有 output,但也会将副本复制到命名的 pipe;
gzip
will read it from the named pipe and compress it. gzip
将从名为 pipe 的文件中读取并压缩它。
You could replace the tee
with the sed
for double-dipping space reduction if required -如果需要,您可以用
sed
替换tee
,以减少双浸空间 -
$: mkfifo transfer
$: gzip < transfer > log.gz &
$: seq 1 100 2>&1 | sed '/.*[37]$/w transfer'
I don't really recommend this, as you might filter out something you didn't realize you would need.我真的不推荐这个,因为你可能会过滤掉你没有意识到你需要的东西。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.