cURL，是curl_multi_init多线程还是异步API？

Question

My question is really simple, is cURL's curl_multi_init actually multi threaded or does it just use an asynchronous API? Thanks!

Answer 1

cURL is a very old library, and truly asynchronous code is a relatively new concept. libcurl was written in C, so every single request is blocking. Although you can process multiple requests in parallel, this is definitely not asynchronous at all for your program as you have to wait until the longest request is finished:

while((curl_multi_exec($master, $running)) == CURLM_CALL_MULTI_PERFORM);

That said, I still believe cURL is the best solution. Other libraries that implement this functionality often also use curl, they just add callbacks so that data can be processed as soon as each individual request is finished, instead of waiting for them all and then reaching 100% CPU to process everything at the end.

The problem is that most implementations of curl_multi wait for each set of requests to complete before processing them. If there are too many requests to process at once, they usually get broken into groups that are then processed one at a time. The problem with this is that each group has to wait for the slowest request to download. In a group of 100 requests, all it takes is one slow one to delay the processing of 99 others. The larger the number of requests you are dealing with, the more noticeable this latency becomes.

The solution is to process each request as soon as it completes. This eliminates the wasted CPU cycles from busy waiting. I also created a queue of cURL requests to allow for maximum throughput. Each time a request is completed, add a new one from the queue. By dynamically adding and removing links, we keep a constant number of links downloading at all times. This gives us a way to throttle the amount of simultaneous requests we are sending. The result is a faster and more efficient way of processing large quantities of cURL requests in parallel.

RollingCurlX Rolling Curl X is a fork of Rolling Curl wrapper cURL Multi. It aims at making concurrent http requests in PHP as easy as possible.
RollingCurl for PHP (not updated in 5 years)

Answer 2

http://php.net/manual/en/function.curl-multi-init.php

Allows the processing of multiple cURL handles asynchronously.