Why does LWP::UserAgent GET request fail with HTTPS?

Question

Here's my code

#!/path/to/perl
use strict;
use LWP::UserAgent;
use HTTP::Request::Common;
use Crypt::SSLeay;

$ENV{HTTPS_PROXY} = 'http://proxy:8080/';

$ENV{HTTPS_DEBUG} = 1;


my $myurl = "https://www.redhat.com";

my $ua = new LWP::UserAgent;
$ua->cookie_jar( {} ); 
$ua->protocols_allowed( [ 'http','https'] );
$ua->proxy(['http', 'https'], 'http://proxy:8080/');

my $page = $ua->get($myurl);

die "Error $myurl\n ", $page->status_line, "\n Aborting" 
unless $page->is_success; 
print "Success", $page1->content_type, " document!\n"; 

It returns

Error at https://www.redhat.com
400 Bad Request
Aborting at test.pl line 30.

what's wrong?

Edit:

Apparently, Its a bug . But the workaround doesn't work for me.

Answer 1

I just uploaded the LWP::Protocol::connect module to CPAN. This module adds the missing HTTP/CONNECT method support to LWP.

 use LWP::UserAgent;

 $ua = LWP::UserAgent->new(); 
 $ua->proxy('https', 'connect://proxyhost.domain:3128/');

 $ua->get('https://www.somesslsite.com');

With this module you can use the regular IO::Socket::SSL implementation for LWP >=6.00.

Answer 2

Ha! I got the answer!

1) remove the '/' after the port of ENV{HTTPS_PROXY}

2) Apparently, LWP's proxy system send 'GET' requests instead of CONNECT requests so use Crypt::SSLeay's proxy system by just setting the environment variable and remove the proxy command.

Answer 3

On some systems, eg Debian, you need to install the appropriate SSL library for this to work. The error messages on theses systems can sometimes be at bit missleading. I think the Debian package would be libnet-ssleay-perl.

Answer 4

It looks like your proxy server does not accept HTTPS connections. Have you tried setting it up in your favorite browser and viewing the URL?