需要用file1的第二列替换与file1的第一列匹配的file2中的字符串

Question

So, if the title didn't make sense, here's what I'm trying to do:

I have file1:

66.115.135.84:123.123.123.1
66.115.135.85:123.123.123.2
66.115.135.86:123.123.123.3
66.115.135.87:123.123.123.4
66.115.135.88:123.123.123.5
66.115.135.89:123.123.123.6
66.115.135.90:123.123.123.7
66.115.135.91:123.123.123.8
66.115.135.92:123.123.123.9
66.115.135.93:123.123.123.10
66.115.135.94:123.123.123.11
66.115.135.95:123.123.123.12
66.115.135.96:123.123.123.13
66.115.135.97:123.123.123.14

As you can see, it's ip addresses, separated by a ":"

File2 is basically an apache virtual host entry, or httpd.conf file. It doesn't really matter. Just know that file2 contains the ip addresses from the first column of file1 somewhere in there. And they need to be replaced by the second column of file1.

For some reason, this simple problem has left me bewildered. I've tried some pretty gnarly things, but keep getting stuck.

I know I can separate them using awk, and i know i could pipe that into sed to act on file2.

But I can't seem to wrap my head around the best way to "map" column 1 to column 2 so that this can actually happen.

I'm willing to use perl, or ruby, or python, or really any method of achieving this, and I would very much like a brief explanation of how you are able to solve this.

Please ask for any clarification, and I'll be glad to supply it.

Thanks so much in advance!

Answer 1

Read the IP pairs from file1 into a hash, eg $ip{$old} = $new . I assume there are no duplicate IPs. Go through file2 looking for IPs, and use a regex such as:

s#($IPregex)# $ip{$1} // $1 #eg;

Code something like:

use autodie;

open my $fh, '<', "file1";
my %ip;
while (<$fh>) {
    chomp;
    my ($key, $val) = split /:/, $_, 2;
    $ip{$key} = $val;
}

open $fh, '<', "file2";
my $rx = qr/\b\d{0,3}\.\d{0,3}\.\d{0,3}\.\d{0,3}\b/;

while (<$fh>) {
    s#($rx)# $ip{$1} // $1 #eg;
    print;
}

Redirect to output file as needed. Might need a better regex for the IP.

Answer 2

perl  -ne '/(.*):(.*)/; (exists $ips{$1}) ? (print "$ips{$1}\n") : ($ips{$1} = $2);' f1 f2

This loops over file f1 then file f2. It splits them on the ':' character and if we haven't seen the first half before, stick it in the hash. If we have seen the first half before, print the value we stored in the hash.

Answer 3

sed -e "s:$(sed -e ':a;$!N;s/\n/:g;s:/g;ta' file1):" file2

The inner sed creates a multi-expression regex for the outer sed to apply to file2..
To securely update your original file insitu, you can pipe the output to ir via sponge (from package moreutils ).

Answer 4

awk '
  FILENAME == ARGV[1] {
    split($0, ary, /:/)
    map[ary[0]] = ary[1]
    next
  }
  {
    for (i=1; i<=NF; i++) {
      if ($i in map)
        $i = map[$i]
    }
    print
  }
' file1 file2 > file2.new

Answer 5

Id use perl. Lets call it mapper.pl . Takes map file as an arg, and then maps stdin to stdout . So you use it like this

perl mapper.pl file1 < file2 > file2.new

The mapper.pl program is something like:

use strict;
use warnings;

# Prototypes
sub readMap($);

# Main program
{
    if( scalar(@ARGV) != 1 )
    {
        die "usage: mapper.pl mapfile";
    }
    my %map = readMap( $ARGV[0] );
    while( my $line = <STDIN> )
    {
        foreach my $old ( keys(%map) )
        {
            my $old_re = $old;
            # Escape metacharacters
            $old_re =~ s/\W/\\$&/g;
            $line =~ s/$old_re/$map{$old}/g;
        }
        print $line;
    }
} # END main

sub readMap($)
{
    my $mapname = $_[0];
    my %map;
    open( MAPFILE, "<$mapname" ) || die "open($mapname): $!";
    while( my $line = <MAPFILE> )
    {
        if( $line =~ /^\s*([^:]+):(.*?)\s*$/ )
        {
            $map{$1} = $2;
        }
        else
        {
            warn "Invalid line: $line";
        }
    }
    close( MAPFILE );
    return( %map );
} # END readMap

Answer 6

Thanks for all of the great answers!

I was inspired by them to create a ruby version: (it could use some work/reduction, it's not very rubyesque yet, but it works)

#!/usr/bin/ruby
#replaces old ips for new ips in virt file
@orig_ips=Array.new
@new_ips=Array.new
File.open("/home/kevin/scripts/ruby_scripts/test.virt", "r").each do |line|
  if line =~ /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/
    @orig_ips.push(line.split.last.chop)
  end
end

File.open("/home/kevin/scripts/ruby_scripts/new_ip_list", "r").each do |line|
  @new_ips.push(line.split.last)
end

f = File.open("/home/kevin/scripts/ruby_scripts/test.virt")
working_file = f.read
for count in 0..@orig_ips.count - 1  do
  old = @orig_ips[count]
  new = @new_ips[count]
  working_file.gsub!(old, new)
end
puts working_file