GET curl request only works in postman but not in php

Question

I'm trying to get html as response but it only works in postman but not in php. In php I get different response like "Something went wrong..."

What I'm missing?

Here is code:

 $curl = curl_init(); curl_setopt_array($curl, array( CURLOPT_URL => "https://www.avnet.com/shop/SearchDisplay?searchTerm=LMK316BJ476ML-T&countryId=apac&deflangId=-1&storeId=715839038&catalogId=10001&langId=-1&sType=SimpleSearch&resultCatEntryType=2&searchSource=Q&searchType=100&avnSearchType=all", CURLOPT_RETURNTRANSFER => true, CURLOPT_ENCODING => "", CURLOPT_MAXREDIRS => 10, CURLOPT_TIMEOUT => 30, CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1, CURLOPT_CUSTOMREQUEST => "GET", CURLOPT_HTTPHEADER => array( "Cache-Control: no-cache" ), )); $response = curl_exec($curl); $err = curl_error($curl); curl_close($curl); if ($err) { echo "cURL Error #:". $err; } else { echo $response; }

postman response

php response

Thanks in advance.

Answer 1

I played around the CURLOPT_HTTPHEADER and figured out the site is validating 2 headers which are User-Agent and Accept-Language. I sent it correctly and it worked. Please find the headers below

CURLOPT_HTTPHEADER => array( "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36", "Accept-Language:en-US,en;q=0.5" )

Enjoyed figuring out the issue.

Answer 2

I had the same problem. This might solve the problem:

 curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);

Answer 3

remove this CURLOPT_ENCODING => "",