curl_multi_add_handle produces an error in file “Unknown” on line 0

Question

I'm trying to use curl_multi_* to fetch multiple (around 50) URLs simultaneously since doing it consecutively takes a long time. However I get the following error message when executing curl_multi_add_handle .

: (null)(): 10 is not a valid cURL handle resource in on line ：（null）（）：10不是第行的的有效cURL句柄资源

Here is the code:

//create the multiple cURL handle
$mh = curl_multi_init();

// Loop over pages and get set the URL to the cURL queue
foreach ($htmltogetlist as $source) {

    [...]

    // Get a cURL handle of the current URL
    $urls[$id]['ch'] = $this->_getCurlHandle($urls[$id]['url']);

    // Success
    if (gettype($urls[$id]['ch']) == 'resource' && get_resource_type($urls[$id]['ch']) == 'curl') {
        curl_multi_add_handle($mh, $urls[$id]['ch']); // << Produces error
    }
}

and $this->_getCurlHandle has:

// Set cURL handle
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'PHP');
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);

return $ch;

What's weird is that $urls[$id]['ch'] is a valid resource of type curl , but yet I can't add it to $mh and the error is in the file unknown on line 0 . Actually, I can even execute the handle and get a correct response from it, so I am positive it's valid. I just can't add it to $mh .

$content = curl_exec($urls[$id]['ch']);
$response = curl_getinfo($urls[$id]['ch']);
print_r($response); // Works

I know that cURL is working for multiple transfers. For example, the following code (which does essentially the same thing) works. I'm also sure that it's not the number of URLs that is causing the problem because I get the same error when limiting the database query to 1:

$ch1 = curl_init();
$ch2 = curl_init();

// set URL and other appropriate options
curl_setopt($ch1, CURLOPT_URL, "http://lxr.php.net/");
curl_setopt($ch2, CURLOPT_URL, "http://www.php.net/");

curl_setopt($ch1,CURLOPT_ENCODING , "gzip");
curl_setopt($ch1, CURLOPT_REFERER, $url);
curl_setopt($ch1, CURLOPT_USERAGENT, 'PHP');
curl_setopt($ch1, CURLOPT_HEADER, true);
curl_setopt($ch1, CURLOPT_AUTOREFERER, true);
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch1, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch1, CURLOPT_TIMEOUT, 10);

curl_setopt($ch2,CURLOPT_ENCODING , "gzip");
curl_setopt($ch2, CURLOPT_REFERER, $url);
curl_setopt($ch2, CURLOPT_USERAGENT, 'PHP');
curl_setopt($ch2, CURLOPT_HEADER, true);
curl_setopt($ch2, CURLOPT_AUTOREFERER, true);
curl_setopt($ch2, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch2, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch2, CURLOPT_TIMEOUT, 10);

//create the multiple cURL handle
$mh = curl_multi_init();

//add the two handles
curl_multi_add_handle($mh,$ch1); // Works
curl_multi_add_handle($mh,$ch2); // Works

Answer 1

I checked the issue #61141 mentioned by Richard in the comments above, and the solution given by pierrick indeed fixed the error for me.

This is the basic multi curl exec setup, as found in the PHP manual:

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) != -1) {
        do {
            $mrc = curl_multi_exec($mh, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);
    }
}

However, the correct setup would be this one:

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) != -1) {
        usleep(100);
    }
    do {
        $mrc = curl_multi_exec($mh, $active);
    } while ($mrc == CURLM_CALL_MULTI_PERFORM);
}

If curl_multi_select returns -1, halt the script for a little while and then run curl_mutli_exec anyway.

For further details: https://bugs.php.net/bug.php?id=61141

Answer 2

I encountered this on OSX with PHP 5.3.23, curl 7.30.0.

The fix was ensuring all the curl_init()'s were called before curl_multi_init();

Never saw this on other platforms (ie Linux)