[英]awk sed perl to merge rows based on keyword match
由于我有限的awk / sed魔法,我在这个问题上一直在撞墙。 我很高兴使用awk,sed,bash,perl或其他任何东西来完成这个文本操作。
我有以下输出,并希望根据一种键匹配合并行:
Node: server1
Active Server: SECONDARY
Standby Server: PRIMARY
Primary 192.168.1.1
Secondary 192.168.1.2
Node: server2
Active Server: PRIMARY
Standby Server: SECONDARY
Primary 10.1.1.1
Secondary 10.1.1.2
期望的输出:
Node: server1
Active Server: Secondary 192.168.1.2
Standby Server: Primary 192.168.1.1
Node: server2
Active Server: Primary 10.1.1.1
Standby Server: Secondary 10.1.1.2
所以我需要根据“primary”和“secondary”这两个词进行合并。 我的第一个想法是将“Primary”改为“PRIMARY”,这样就更容易匹配了。
我最终的目标是拥有:
server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standy,10.1.1.2
(但是我可以在帮助合并行之后弄清楚这部分)
谢谢您的帮助!
这个Perl解决方案似乎可以满足您的要求。 它只是将值逐行拉入哈希值,并在存在所有必需值时转储哈希内容。
更新我使用了List::Util
中的any
代替grep
来使代码更清晰。
use strict;
use warnings;
use autodie;
use List::Util 'any';
my @names = qw/ node active standby primary secondary /;
open my $fh, '<', 'myfile.txt';
my %server;
while (my $line = <$fh>) {
next unless my ($key, $val) = lc($line) =~ /(\w+).*\s+(\S+)/;
%server = () if $key eq 'server';
$server{$key} = $val;
unless ( any { not exists $server{$_} } @names ) {
printf "%s,Active,%s,Standby,%s\n", @server{'node', $server{active}, $server{standby}};
%server = ();
}
}
产量
server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,10.1.1.2
它是密集且非常难看的多线程,
perl -00 -nE'
s/ ^(\w+)\s+([\d.]+)\s* / $s{$1}=$2; ""/xmge;
($l=$_) =~ s! \s*\w+:\s*|\n !,!xg;
$l =~ s|\U$_|$s{$_}| for keys %s;
($_=$l) =~ s/^,|,$//g;
say
' file
产量
server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,10.1.1.2
说明
# -00 => instead of single line read lines into $_ until \n\n+
perl -00 -nE'
# read and remove 'Primary|Secondary IP' into $s{Primary} = IP
s/ ^(\w+)\s+([\d.]+)\s* / $s{$1}=$2; ""/xmge;
# replace 'something:' or new line by ','
($l=$_) =~ s! \s*\w+:\s*|\n !,!xg;
# replace SECONDARY|PRIMARY with actual IP address
$l =~ s|\U$_|$s{$_}| for keys %s;
# remove ',' at beginning and end of the string
($_=$l) =~ s/^,|,$//g;
# print result
say
' file
或者使用单线程用于中间期望的解决方案(最终解决方案):
perl -00 -lpe '
s/ Server: \K(\w+)(?=.*^(\1[^\n]*))/$2/ismg;
s/\n[^:]+$//;
' file.txt
输出:
Node: server1
Active Server: Secondary 192.168.1.2
Standby Server: Primary 192.168.1.1
Node: server2
Active Server: Primary 10.1.1.1
Standby Server: Secondary 10.1.1.2
说明:
-00
:段落模式下的进程输入(由双返回分隔) -l
:启用行结束处理 -p
:假设"while (<>) { ...; print; }"
循环程序 -e
:评估perl代码 为了获得您想要的最终解决方案,以下一个班轮将实现该目标。
第一个解决方案有一些细微的变化,例如使用-n
而不是-p
因为我们希望从记录之间的两个换行符移动到一个新行。 但是,正则表达式工具是相同的:
perl -00 -ne'
s/ Server: (\w+)(?=.*^\1\s+(\S+))/:$2/ismg;
s/\n[^:]+$//;
s/^Node: //;
s/[\n:]/,/g;
print "$_\n";
' file.txt
输出:
server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,10.1.1.2
awk '
$1 == "Active" {active = tolower($NF); next}
$1 == "Standby" {standby = tolower($NF); next}
$1 == "Primary" {ip["primary"] = $0; next}
$1 == "Secondary" {
ip["secondary"] = $0
print "Active Server:",ip[active]
print "Standby Server:",ip[standby]
next
}
1
'
这假定“辅助”行位于“块”的末尾。
要实现下一个输出:
awk -v OFS="," '
$1 == "Node:" {node = $NF}
$1 == "Active" {active = tolower($NF)}
$1 == "Standby" {standby = tolower($NF)}
$1 == "Primary" {ip["primary"] = $2}
$1 == "Secondary" {
ip["secondary"] = $2;
print node, "Active",ip[active],"Standup",ip[standby]
}
'
回应jhill的评论:
awk -v RS="" -v OFS=, '{
node = active = standby = ""
delete ip
for (i=1; i<NF; i++) {
if ($i == "Node:") {node=$(++i)}
else if ($i == "Active") {active = tolower( $(i+=2) )}
else if ($i == "Standby") {standby = tolower( $(i+=2) )}
else if ($i == "Primary") {ip["primary"] = $(++i)}
else if ($i == "Secondary") {ip["secondary"] = $(++i)}
}
print node, "Active", ip[active], "Standup", ip[standby]
}'
您可以使用tr
消除空间,然后sed
放回去,然后在正确的地方,并使用perl
得到你想要的输出:
输入文件:
tiago@dell:/tmp$ cat file
Node: server1
Active Server: SECONDARY
Standby Server: PRIMARY
Primary 192.168.1.1
Secondary 192.168.1.2
Node: server2
Active Server: PRIMARY
Standby Server: SECONDARY
Primary 10.1.1.1
Secondary 10.1.1.2
脚本:
tiago@dell:/tmp$ cat test.sh
#! /bin/bash
tr -d '\n' < $1 | sed -r 's/(Node:)/\n\1/g' |\
perl -lne '
/^\s+$/ && next;
/Node:\s+(\w+.*?)\s/ && {$server=$1};
/Active Server:\s+(\w+.*?)\s/ && {$active=$1};
/Standby Server:\s+(\w+.*?)\s/ && {$standby=$1};
/Primary\s+(\w+.*?)\s/ && {$pri=$1};
/Secondary\s+(\w+.*?)\s/ && {$sec=$1};
if ( "$active" eq "PRIMARY" ){
$out="$server,Active,$pri,Standby,$sec";
}else{
$out="$server,Active,$sec,Standby,$pri";
}
print $out;
'
执行:
tiago@dell:/tmp$ bash test.sh file
server1,Active,192.168.1.2,Standby,192.168.1.1
server2,Active,10.1.1.1,Standby,192.168.1.2
你可以使用这个awk
awk -v RS="" '{$5=tolower($5);sub(".",substr(toupper($5),1,1),$5);$8=tolower($8);sub(".",substr(toupper($8),1,1),$8);print $1,$2"\n"$3,$4,$5,$10"\n",$6,$7,$8,$12}' file
Node: server1
Active Server: Secondary 192.168.1.1
Standby Server: Primary 192.168.1.2
Node: server2
Active Server: Primary 10.1.1.1
Standby Server: Secondary 10.1.1.2
通过sette将RS
设置为空, awk
与行组一起工作。
更冗长一点:
use strict;
use warnings;
use feature qw/say/;
my $struct;
local $/ = 'Node: ';
for my $record (<DATA>) {
next if $record =~ /^Node:/; # skip first
my ($node, @values) = split /\n\s*/, $record;
for my $line (@values) {
my ($intent, $actual, $ip);
if ( ($intent, $actual) = $line =~ /(Active|Standby) Server: (.*)$/ ) {
$struct->{$node}{lc($intent)} = lc($actual);
}
elsif ( ($actual, $ip) = $line =~ /(Primary|Secondary) (.*)$/ ) {
$struct->{$node}{lc($actual)} = $ip;
}
}
}
for my $node (sort keys %$struct) {
printf "Node: %s\n", $node;
printf "Active server: %s %s\n", ucfirst $struct->{$node}{active}, $struct->{$node}{$struct->{$node}{active}};
printf "Standby server: %s %s\n", ucfirst $struct->{$node}{standby}, $struct->{$node}{$struct->{$node}{standby}};
print "\n";
}
## Desired final output is simpler:
for my $node (sort keys %$struct) {
say join ',', $node, 'Active', $struct->{$node}{$struct->{$node}{active}}, 'Standby', $struct->{$node}{$struct->{$node}{standby}};
}
__DATA__
Node: server1
Active Server: SECONDARY
Standby Server: PRIMARY
Primary 192.168.1.1
Secondary 192.168.1.2
Node: server2
Active Server: PRIMARY
Standby Server: SECONDARY
Primary 10.1.1.1
Secondary 10.1.1.2
这是awk中的一个选项。
#!/usr/bin/awk -f
# Output processing goes in a function, as it's called from different places
function spew() {
split(servers[d["active"]], active);
split(servers[d["standby"]], standby);
printf("%s,%s,%s,%s,%s\n",
d["name"], active[1], active[2], standby[1], standby[2]);
}
# trim unnecessary (leading) whitespace
1 { $1=$1; }
# Store our references
$1=="Active" {
d["active"]=tolower($3);
}
#
$1=="Standby" {
d["standby"]=tolower($3);
}
# And store our data
/^ *[A-za-z]+ [0-9.]+$/ {
servers[tolower($1)]=tolower($0);
}
# Then, if we hit a new record, process the last one.
$1=="Node:" && length(d["name"]) {
spew();
}
# And if we've just process a record, clear our workspace.
$1=="Node:" {
delete d;
delete s;
d["name"]=$2;
}
# Finally, process the last record.
END {
spew();
}
与其他一些解决方案相比,它的一个优点是它可以处理除“主要”和“次要”之外的名称。 这个想法是,如果你有这样的数据:
Node: serverN
Active Server: starfleet
Standby Server: babylon5
starfleet 172.16.0.1
babylon5 172.16.0.2
活动/备用线路将通过其索引引用记录,而不是假设为“主要”或“次要”。
我已将所有内容标准化为小写以便于处理,但您当然可以调整tolower()
以适应。
awk ' s==0{print;s=1;next;}
s==1{i=$0;s=2;next;}
s==2{j=$0;s=3;next;}
s==3{r1=$0;s=4;next;}
s==4{r2=$0;
sub(/SECONDARY/,r2,i);sub(/PRIMARY/,r1,j);
sub(/SECONDARY/,r2,j);sub(/PRIMARY/,r1,i);
s=5; print i;print j;next}
s==5{s=0;print}' input.txt
输出:
Node: server1
Active Server: Secondary 192.168.1.2
Standby Server: Primary 192.168.1.1
Node: server2
Active Server: Primary 10.1.1.1
Standby Server: Secondary 10.1.1.2
打印当前输入节的第一行,将下四行存储在变量中,然后进行替换,然后打印结果。 然后读取并打印空行并再次开始下一部分。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.