Perl/ curl How to get Status Code and Response Body

Question

I am trying to write a simple perl script that calls and API and if the status code is 2xx the do something with the response. While if it is 4xx or 5xx then do something else.

The issue I am encountering is I am able to either get the response code (using a custom write-out formatter and pass the output somewhere else) or I can get the whole response and the headers.

my $curlResponseCode = `curl -s -o /dev/null -w "%{http_code}" ....`;

Will give me the status code only.

my $curlResponse = `curl -si ...`; 

Will give me the entire header plus the response.

My question is how can I obtain the response body from the server and the http status code in a neat format that allows me to separate them into two separate variables.

Unfortunately I cannot use LWP or any other separate libraries.

Thanks in advance. -Spencer

Answer 1

I came up with this solution:

URL="http://google.com"

# store the whole response with the status at the and
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST $URL)

# extract the body
HTTP_BODY=$(echo $HTTP_RESPONSE | sed -e 's/HTTPSTATUS\:.*//g')

# extract the status
HTTP_STATUS=$(echo $HTTP_RESPONSE | tr -d '\n' | sed -e 's/.*HTTPSTATUS://')

# print the body
echo "$HTTP_BODY"

# example using the status
if [ ! $HTTP_STATUS -eq 200  ]; then
  echo "Error [HTTP status: $HTTP_STATUS]"
  exit 1
fi

Answer 2

...Will give me the entire header plus the response.

...in a neat format that allows me to separate them into two separate variables.

Since header and body are simply delimited by an empty line you can split the content on this line:

 my ($head,$body) = split( m{\r?\n\r?\n}, `curl -si http://example.com `,2 );

And to get the status code from the header

 my ($code) = $head =~m{\A\S+ (\d+)};

You might also combine this into a single expression with a regexp, although this might be harder to understand:

my ($code,$body) = `curl -si http://example.com` 
      =~m{\A\S+ (\d+) .*?\r?\n\r?\n(.*)}s;

Answer 3

Pretty fundamentally - you're capturing output from a system command. It is far and away better to do this by using the library built for it - LWP . Failing that though - curl -v will produce status code and content, and you'll have to parse it.

You might also find this thread on SuperUser useful:

https://superuser.com/questions/272265/getting-curl-to-output-http-status-code

Specifically

#creates a new file descriptor 3 that redirects to 1 (STDOUT)
exec 3>&1 
# Run curl in a separate command, capturing output of -w "%{http_code}" into HTTP_STATUS
# and sending the content to this command's STDOUT with -o >(cat >&3)
HTTP_STATUS=$(curl -w "%{http_code}" -o >(cat >&3) 'http://example.com')

(That isn't perl, but you can probably use something similar. At very least, running the -w and capturing your content to a temp file.

Answer 4

Haven't figured out a "pure" Perl solution, but I drafted this snippet to check the HTTP response code of a page via curl :

#!/usr/bin/perl

use v5.30;

use warnings;
use diagnostics;

our $url = "";

my $username = "";
my $password = "";

=begin url_check
Exit if HTTP response code not 200.
=cut

sub url_check {

  print "Checking URL status code...\n";

  my $status_code =
(`curl --max-time 2.5 --user ${username}:${password} --output /dev/null --silent --head --write-out '%{http_code}\n' $url`);

  if ($status_code != '200'){
    {
      print "URL not accessible. Exiting. \n";
      exit;
    }
  } else {
      print "URL accessible. Continuing... \n";
    }
}

url_check

The verbose use of curl more or less documents itself. My example allows you to pass credentials to a page, but that can be removed as needed.