Can anyone show me how to compose this request using curl php? I need to scrape a website, but the code I use (given below) always returns a blank page. I want to send
Accept-Encoding, Useragent, and cookie
Request:
Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.99 Safari/535.1
Accept-Encoding: gzip
to
Thanks!
The code I tried to use (returned a blank page):
<?
$url='https://example.com';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request_headers);
$request_headers = array();
$request_headers[] = 'Accept-Encoding: gzip';
$request_headers[] = 'Client: Apple';
curl_setopt($ch, CURLOPT_COOKIE, 'insertedmycookiehere');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.99 Safari/535.1');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
?>
Did I do something wrong?
This worked for me :
$url='http://example.com'; /* You incorrectly used https for this website */
$ch = curl_init($url);
$request_headers = array();
$request_headers[] = 'Accept-Encoding: gzip';
$request_headers[] = 'Client: Apple';
curl_setopt($ch, CURLOPT_HTTPHEADER, $request_headers); /* Moved this line here */
curl_setopt($ch, CURLOPT_COOKIE, 'insertedmycookiehere');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.99 Safari/535.1');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.