[英]Perl: STDOUT/the output of shell command to an array directly
I have to access a shell command - hive within a Perl script, So I use `...`. 我必须访问一个shell命令-Perl脚本中的hive,所以我使用`...`。 Assuming the result of `hive ... ...` contains 100000000 lines and is 20GB size. 假设“ hive ... ...”的结果包含100000000行,大小为20GB。 what I want to achieve is like this: 我想要实现的是这样的:
@array = `hive ... ...`;
Does `` automatically know to use "\\n" as separator to divide each line into the @array? ``是否自动知道使用“ \\ n”作为分隔符将每一行划分为@array?
The 2 ways I can thought of are (but with problem in this case): 我可以想到的2种方法是(但在这种情况下有问题):
$temp = `hive ... ...`;
@array = split ( "\n", $temp );
undef $temp;
The problem of this way is that if the output of hive is too big in this case, the $temp cant store the output, resulting in segmentation fault core dump. 这种方式的问题是,如果在这种情况下hive的输出太大,则$ temp无法存储输出,从而导致分段故障核心转储。
OR 要么
`hive ... ... 1>temp.txt`;
open ( FP, <, "temp.txt" );
while (<FP>)
{
chomp;
push @array, $_;
}
close FP;
`rm temp.txt`;
But this way would be too slow, because it writes result first to hard-disk. 但是这种方式太慢了,因为它首先将结果写入硬盘。
Is there a way to write the output of a shell command directly to an array without using any 'temporary container'? 有没有一种方法可以在不使用任何“临时容器”的情况下直接将shell命令的输出写入数组?
Very Thanks for helping. 非常感谢您的帮助。
@array = `command`;
does, in fact, put each line of output from command
into its own element of @array
. 实际上,确实将command
的输出的每一行都放在其自己的@array
元素中。 There is no need to load the output into a scalar and split
it yourself. 无需将输出加载到标量并将其自己split
。
But 20GB of output stored in an array (and possibly 2-3 times that amount due to the way that Perl stores data) will still put an awful strain on your system. 但是,存储在阵列中的20GB输出(由于Perl存储数据的方式可能是该输出的2-3倍)仍然会对您的系统造成很大的压力。
The real solution to your problem is to stream the output of your command through an IO handle, and deal with one line at a time without having to load all of the output into memory at once. 解决问题的真正方法是通过IO句柄流式传输命令的输出,并一次处理一行,而不必一次将所有输出加载到内存中。 The way to do that is with Perl's open
command: 使用Perl的open
命令可以做到这一点:
open my $fh, "-|", "command";
open my $fh, "command |";
The -|
-|
filemode or the |
文件模式或|
appended to the command tells Perl to run an external command, and to make the output of that command available in the filehandle $fh
. 附加在命令后的命令告诉Perl运行外部命令,并在文件句柄$fh
提供该命令的输出。
Now iterate on the filehandle to receive one line of output at a time. 现在,在文件句柄上进行迭代以一次接收一行输出。
while (<$fh>) {
# one line of output is now in $_
do_something($_);
}
close $fh;
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.