Finding a value in HTTP response content using Perl Script

Question

I have perl script with HTTP GET request. My repsonse content is like

$VAR1 = \'{"ResultSet": {
  "result": [
    {
      "rank": "999999",
      "term": "shampoo"
    },
    {
      "rank": "999999",
      "term": "Beauty",
      "url": "/search/results.jsp?Ntt=shampoo&N=359434"
    },
    {
      "rank": "999999",
      "term": "Baby, Kids & Toys",
      "url": "/search/results.jsp?Ntt=shampoo&N=359449"
    },

I need url property from above response how can i get it. Itried using regex like my $content =~ m/:"url": "(...)"/; but i am not getting the url value. Please guide.

Answer 1

That is JSON. So use the JSON module to parse it:

use JSON; 
my $json = decode_json ( $response -> content ); 
foreach my $element ( @{ $json -> {ResultSet} -> {results} } ) {
    print $element -> {url},"\n"; 
}

Fuller; runnable example:

#!/usr/bin/perl

use strict;
use warnings;
use JSON;
use Data::Dumper;

my $json_str = '{
  "ResultSet": {
  "result": [
    {
      "rank": "999999",
      "term": "shampoo"
    },
    {
      "rank": "999999",
      "term": "Beauty",
      "url": "/search/results.jsp?Ntt=shampoo&N=359434"
    },
    {
      "rank": "999999",
      "term": "Baby, Kids & Toys",
      "url": "/search/results.jsp?Ntt=shampoo&N=359449"
    }
  ]
}}';

my $json = decode_json($json_str);
print Dumper $json;
foreach my $element ( @{ $json->{ResultSet}->{result} } ) {
    print $element ->{url}, "\n" if $element->{url};
}

In the above, $json_str fills the niche of your content. I've made the assumption that you have plain text, and the output above is the result of print Dumper \\$content .

This thus prints:

$VAR1 = {
          'ResultSet' => {
                           'result' => [
                                         {
                                           'rank' => '999999',
                                           'term' => 'shampoo'
                                         },
                                         {
                                           'rank' => '999999',
                                           'term' => 'Beauty',
                                           'url' => '/search/results.jsp?Ntt=shampoo&N=359434'
                                         },
                                         {
                                           'url' => '/search/results.jsp?Ntt=shampoo&N=359449',
                                           'term' => 'Baby, Kids & Toys',
                                           'rank' => '999999'
                                         }
                                       ]
                         }
        };

/search/results.jsp?Ntt=shampoo&N=359434
/search/results.jsp?Ntt=shampoo&N=359449

Answer 2

You have a reference to a JSON string.

---

First, get the JSON.

my $json = $$content;

If you (incorrectly) did Dumper(\\$content) instead of Dumper($content) , then ignore the above and use the following instead:

my $json = $content;   # Or just use $content where you see $json later.

---

Then, use a JSON parse to get the data.

use JSON::XS qw( decode_json );
my $data = decode_json($json);             # If the $json is UTF-8 (probably)
  -or-
use JSON::XS qw( );
my $data = JSON::XS->new->decode($json);   # If the $json is decoded (Unicode Code Points)

---

Now, it's easy to grab your data.

my $results = $data->{ResultSet}{result};

for my $result (@$results) {
   my $url = $result->{url}
      or next;

   print("$url\n");
}