[英]How can I use cURL to open multiple URLs simultaneously with PHP?
這是我當前的代碼:
$SQL = mysql_query("SELECT url FROM urls") or die(mysql_error()); //Query the urls table
while($resultSet = mysql_fetch_array($SQL)){ //Put all the urls into one variable
// Now for some cURL to run it.
$ch = curl_init($resultSet['url']); //load the urls
curl_setopt($ch, CURLOPT_TIMEOUT, 2); //No need to wait for it to load. Execute it and go.
curl_exec($ch); //Execute
curl_close($ch); //Close it off
} //While loop
我對cURL比較陌生。 相對而言,這是我第一次使用cURL。 當前,它加載一個持續兩秒鍾,然后加載下一個持續2秒,然后加載下一個。 但是,我想使其同時加載所有這些。 我確定這是可能的,我不確定如何做到。 如果有人能指出我正確的方向,我將不勝感激。
您可以用相同的方式設置每個cURL句柄,然后將它們添加到curl_multi_
句柄中。 要查看的函數是此處記錄的curl_multi_*
函數。 不過,根據我的經驗,嘗試一次加載太多URL時存在一些問題(盡管目前無法在其上找到我的注釋),因此上次使用curl_mutli_
,我將其設置為可以批量處理一次5個網址。
編輯 :這是我使用curl_multi_
的代碼的簡化版本:
編輯 :稍微重寫和大量添加的注釋,希望能有所幫助。
// -- create all the individual cURL handles and set their options
$curl_handles = array();
foreach ($urls as $url) {
$curl_handles[$url] = curl_init();
curl_setopt($curl_handles[$url], CURLOPT_URL, $url);
// set other curl options here
}
// -- start going through the cURL handles and running them
$curl_multi_handle = curl_multi_init();
$i = 0; // count where we are in the list so we can break up the runs into smaller blocks
$block = array(); // to accumulate the curl_handles for each group we'll run simultaneously
foreach ($curl_handles as $a_curl_handle) {
$i++; // increment the position-counter
// add the handle to the curl_multi_handle and to our tracking "block"
curl_multi_add_handle($curl_multi_handle, $a_curl_handle);
$block[] = $a_curl_handle;
// -- check to see if we've got a "full block" to run or if we're at the end of out list of handles
if (($i % BLOCK_SIZE == 0) or ($i == count($curl_handles))) {
// -- run the block
$running = NULL;
do {
// track the previous loop's number of handles still running so we can tell if it changes
$running_before = $running;
// run the block or check on the running block and get the number of sites still running in $running
curl_multi_exec($curl_multi_handle, $running);
// if the number of sites still running changed, print out a message with the number of sites that are still running.
if ($running != $running_before) {
echo("Waiting for $running sites to finish...\n");
}
} while ($running > 0);
// -- once the number still running is 0, curl_multi_ is done, so check the results
foreach ($block as $handle) {
// HTTP response code
$code = curl_getinfo($handle, CURLINFO_HTTP_CODE);
// cURL error number
$curl_errno = curl_errno($handle);
// cURL error message
$curl_error = curl_error($handle);
// output if there was an error
if ($curl_error) {
echo(" *** cURL error: ($curl_errno) $curl_error\n");
}
// remove the (used) handle from the curl_multi_handle
curl_multi_remove_handle($curl_multi_handle, $handle);
}
// reset the block to empty, since we've run its curl_handles
$block = array();
}
}
// close the curl_multi_handle once we're done
curl_multi_close($curl_multi_handle);
鑒於您不需要從URL返回任何內容,因此您可能不需要很多東西,但這就是我將請求分塊到BLOCK_SIZE
塊中的BLOCK_SIZE
,等待每個塊運行然后繼續前進,然后捕獲cURL錯誤。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.