curl_multi_exec performance conern

Question

The following example is taken from:

http://php.net/manual/en/function.curl-multi-close.php#example-3540

This example will create two cURL handles, add them to a multi handle, and then run them in parallel.

<?php
// create both cURL resources
$ch1 = curl_init();
$ch2 = curl_init();

// set URL and other appropriate options
curl_setopt($ch1, CURLOPT_URL, "http://www.example.com/");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_URL, "http://www.php.net/");
curl_setopt($ch2, CURLOPT_HEADER, 0);

//create the multiple cURL handle
$mh = curl_multi_init();

//add the two handles
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);

$running=null;
//execute the handles
do {
    curl_multi_exec($mh,$running);
} while ($running > 0);

//close the handles
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);

?>

I adjusted it a bit to achieve my needs. I want to call only one resource but execute local code while it is requested. This works just fine and my performance concern is also valid for the example without modification.

I produced resources that take 5 seconds to deliver and used them as $ch1 and $ch2

As expected the total execution took only 5 seconds instead of 10.

But in the do loop I added a small counter that revealed that curl_multi_exec and respectively the do loop get executed ~5000000 times on my local machine during the ~5 seconds. That would be about one run per microsecond, which is really quite a lot.

I am worried that this occupies extensive CPU resources while just waiting for the requests to finish and basically doing nothing.

I remembered that one of the first things I learned at a Java course was to avoid endless loops and work with Thread sleeps/interrupts instead for the reasons stated above.

However we don't have threads here and I don't know whether this is valid for PHP too.

As this is an official example I thought I rather ask the experts here first.

I thought about implementing a short sleep in the do loop. Somehing like usleep(100) .

• Is my concern valid?

  • If so, is the sleep solution a good one?

    • If so, what would be a good sleep interval? I would love something as small as possible so no process gets slowed down, but I also don't want to hammer the server.

  • If not, why?

Answer 1

Your program loops like crazy like this because curl_exec_multi() will always execute and try to do something and your program doesn't wait accordingly for something to act on before you call it again.

See curl_multi_select for the PHP function that is explicitly made for this purpose and should solve your problem.