cURL 爬虫拒绝访问 PHP

Question

I'm creating a crawler to capture some public information. However, it is returning:

Access Denied
You don't have permission to access "http://www.americanas.com.br/" on this server.

Using Postman to test a request, cURL works perfectly. I even got the code generated by Postman (as shown below), but when I use it directly on my PHP server, return the error informed above.

My cURL code:

$curl = curl_init();

curl_setopt_array($curl, array(
    CURLOPT_URL => "https://www.americanas.com.br/",
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "GET",
    CURLOPT_HTTPHEADER => array(
        "cache-control: no-cache",
        "postman-token: 112ebf89-1bb7-aa7a-0645-cdeabcf96488"
    ),
));

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if($err) echo "cURL Error #:" . $err;
else echo $response;
exit();

Answer 1

I found that there are sites with more complex locks. In these cases, it is necessary to use more complete crawler solutions. The one I'm using and is working is Proxycawl ( https://proxycrawl.com/ ).

Answer 2

Your postman is querying https ://www.americanas.com.br/ while from the error message we can suppose that in your crawler you are querying http ://www.americanas.com.br/