CURL works in command line but not in the PHP script

Question

It's already asked here , but the asker didn't give enough related log and code so I couldn't use it to fix my problem. There is no ampersand (&) in the cookie, so it's certainly not the problem.

So, I can successfully open a web page using curl in command line using this command string:

curl "https://example.com" -H "cookie: wpfront-notification-bar-landingpage=1; gsScrollPos=; PHPSESSID=qhdkf5id0mnjcjadeuadsfkfp3; _gat=1; arp_scroll_position=877; _drip_client_1350063=vid^%^253D57a7eb508b0f013477ed126e9005972f^%^2526pageViews^%^253D8^%^2526sessionPageCount^%^253D2^%^2526lastVisitedAt^%^253D1478973050670^%^2526weeklySessionCount^%^253D2^%^2526lastSessionAt^%^253D1478972975034^%^2526form^%^255B2550^%^255D^%^255Bauto_open^%^255D^%^253D1478960007^%^2526form^%^255B2550^%^255D^%^255Bmanual_close^%^255D^%^253D1478960015; _ga=GA1.2.660807988.1478959677"

But my PHP script which should work exactly the same, doesn't work. I tried adding cookie to header and also adding cookie right away, but it's just the same.

function request($url) {
    $ch = curl_init();
    //curl_setopt($ch, CURLOPT_TIMEOUT_MS, 2000);
    curl_setopt_array($ch, array(
        CURLOPT_URL => $url,
        CURLOPT_SSL_VERIFYPEER      =>  0,
        CURLOPT_RETURNTRANSFER      =>  1,
        CURLOPT_CONNECTTIMEOUT      =>  10,
        CURLOPT_TIMEOUT             =>  10,
        CURLOPT_ENCODING            =>  "gzip",
        CURLOPT_FOLLOWLOCATION      =>  TRUE,
        CURLOPT_HEADER              =>  TRUE,
        CURLOPT_USERAGENT           =>  "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36",
        CURLOPT_HTTPHEADER          => array("cookie: wpfront-notification-bar-landingpage=1; gsScrollPos=; PHPSESSID=qhdkf5id0mnjcjadeuadsfkfp3; _gat=1; arp_scroll_position=877; _drip_client_1350063=vid^%^253D57a7eb508b0f013477ed126e9005972f^%^2526pageViews^%^253D8^%^2526sessionPageCount^%^253D2^%^2526lastVisitedAt^%^253D1478973050670^%^2526weeklySessionCount^%^253D2^%^2526lastSessionAt^%^253D1478972975034^%^2526form^%^255B2550^%^255D^%^255Bauto_open^%^255D^%^253D1478960007^%^2526form^%^255B2550^%^255D^%^255Bmanual_close^%^255D^%^253D1478960015; _ga=GA1.2.660807988.1478959677"),
        CURLOPT_VERBOSE             => 1,
    ));
    $response = curl_exec($ch);
    $header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
    $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    $header = substr($response, 0, $header_size);
    $html = substr($response, $header_size);

    echo $html;

    libxml_use_internal_errors(TRUE);
    $dom = new DOMDocument();
    if (strpos($url, 'feed') !== false) {
        $dom->loadXML($html);
    } else {
        $dom->loadHtml($html);
    }
    return new DOMXPath($dom);
    libxml_clear_errors(); // to clear memory up
}

request("http://example.com");

How can I successfully run this curl via a PHP script? Why does it work in the command line but not in the PHP script? Is there anything I do wrong?

Any help would be appreciated.

Answer 1

I solve it by using shell_exec() instead.

echo shell_exec("curl https://example.com");

It seems there is a bug with the curl library combined with a bug in the site's page. The bug is not that consistent. Sometimes the command line curl requires cookies, but mostly not.

Well, I don't have time to debug curl, so if shell_exec() works, I am okay with it. It even becomes much simpler.