CURL PHP 爬虫返回访问被拒绝错误

Question

My dad loves these frozen cheeseburgers from meijer so I was gonna write a little script I can run in cron that will check Meijer's website and txt or email or something if they go on sale.

Whenever I run the below script I get an Access Denied response from the server instead of the html for the cheesburger page.

I'm sure I just need a CURL option or something.

Thank You in Advance

function curl_download($Url)
{
    if (!function_exists('curl_init')){die('cURL is not installed. Install and try again.');}

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $Url);

    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_VERBOSE, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_HEADER, true);

    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0");

    $output = curl_exec($ch);
    curl_close($ch);

        return $output;
}

print curl_download("https://www.meijer.com/shop/en/frozen/frozen-meals/sandwiches/meijer-bacon-cheeseburger-4-9-oz/p/71373326278");

Answer 1

You very likely have a few more header settings to add before the server is unable to detect that you are using curl and therefore not a regular human user.

Can servers block curl requests?

Some websites have even more advanced techniques to determine whether or not you are a human user, such as whether or not JavaScript loaded on an earlier page, but making sure your header variables match an actual user request is necessary.

you can use the Google Chrome or Firefox inspector to see what request headers you should be sending.