Linux: How to replace a string of text containing quotes and commas in a file

Question

I'm not the greatest with awk, perl, or sed, and I can't figure out how to replace a string of text in a file.

The string is somewhat unusual. Actually, there are 2 of them I need to replace in 1 file automatically.

The file contains a single line of text, very long, in JSON format. Per-line replacements won't work.

Here is the text I need to replace. From:

"minimumVersion":2,"librariesLocation":"libraries","objectsLocation":"objects",

to

"minimumVersion":2,"librariesLocation":"pack/libraries","objectsLocation":"pack/objects",

As you can see there are quotes in the text and slashes I need to enter, as well as commas and multiple occurences of libraries and objects, making my current knowledge of sed, awk, and perl useless.

How can I add the 'pack/' prefix to the text 'libraries' and 'objects'?

Answer 1

Please don't try and use regex for JSON. It's nasty. Use a JSON parser:

#!/usr/bin/env perl

use strict;
use warnings;
use JSON; 

my $json_str = '{ "minimumVersion":2,"librariesLocation":"libraries","objectsLocation":"objects" }';

my $json_ob = decode_json($json_str);

print "Before:", $json_ob -> {'librariesLocation'},"\n";

#pattern replace just the values we're interested in.
$json_ob -> {'librariesLocation'} =~ s,^,pack/,;
$json_ob -> {'objectsLocation'} =~ s,^,pack/,;

print "After: ", $json_ob -> {'librariesLocation'},"\n";

#print single line text blob
print encode_json($json_ob);
print "\n\n";

## or perhaps (formatted, multiline, whitespace):

print to_json($json_ob, { canonical => 1, pretty => 1 } );

This works with the rough sample - if you want a better one, give me more JSON and I'll redraft.

This outputs:

Before:libraries
After: pack/libraries
{"objectsLocation":"pack/objects","minimumVersion":2,"librariesLocation":"pack/libraries"}

{
   "librariesLocation" : "pack/libraries",
   "minimumVersion" : 2,
   "objectsLocation" : "pack/objects"
}

Edit: To do this from a file:

local $/; 
open ( my $input, '<', 'source_filename_here' ) or die $!;
open ( my $output, '>', 'output_filename_here' ) or die $!;

my $json_ob = decode_json(<$input>);
#do the transforms

print {$output} encode_json($json_ob); 

Or alternatively (reading from STDIN printing to STDOUT just like sed/grep etc.):

#!/usr/bin/env perl

use strict;
use warnings;
use JSON;

my @stuff_to_change = qw ( librariesLocation objectsLocation );

local $/;

#read from stdin or arg on commmand line. E.g.:
#myscript.pl somefile
#cat json_file | myscript.pl
my $json_ob = decode_json(<>);

#pattern replace just the values we're interested in.
for my $thing (@stuff_to_change) {
    $json_ob->{$thing} =~ s,^,pack/,;
}

#print single line text blob to STDOUT
print encode_json($json_ob);

Answer 2

How can I add the 'pack/' prefix to the text 'libraries' and 'objects'?

How about a simple sed command:

echo '"minimumVersion":2,"librariesLocation":"libraries","objectsLocation":"objects",' > file

sed 's~"\(libraries\|objects\)"~"pack/\1"~g' file

Output:

"minimumVersion":2,"librariesLocation":"pack/libraries","objectsLocation":"pack/objects",

---

To run over ssh:

ssh ip<<'EOF'
sed -i 's~"libraries"~"pack/&"~; s~"objects"~"pack/&"~' file.json 
EOF

Answer 3

sed --in-place will do the trick, with the correct escaping

$ echo '"minimumVersion":2,"librariesLocation":"libraries","objectsLocation":"objects",' > text.txt      
$ sed --in-place 's/"\(libraries\|objects\)"/"pack\/\1"/g' text.txt
$ cat text.txt 
  "minimumVersion":2,"librariesLocation":"pack/libraries","objectsLocation":"pack/objects",

replace pattern "libraries" with "pack/libraries" (using grouping and reuse of matching pattern)

Answer 4

With proper JSON input, you could use jq :

$ jq '.librariesLocation |= "pack/"+., .objectsLocation |= "pack/"+.' <<EOF
{"minimumVersion":2,"librariesLocation":"libraries","objectsLocation":"objects"}
EOF
{
  "minimumVersion": 2,
  "librariesLocation": "pack/libraries",
  "objectsLocation": "objects"
}
{
  "minimumVersion": 2,
  "librariesLocation": "libraries",
  "objectsLocation": "pack/objects"
}

If your input is just a comma-separated list of key/value pairs, you might be able to use jq to turn it into a proper JSON object before filtering, but I don't know jq well enough yet to figure out how.

Answer 5

awk '{gsub(/libraries",|objects",/, "pack/&")}1' file
minimumVersion":2,"librariesLocation":"pack/libraries","objectsLocation":"pack/objects",