[英]Multithreading or Parallel processing in PHP
我正在處理Godaddy拍賣域名,它們提供了一些下載域名列表的方法。 我確實開發了一項cron作業,以將域列表下載並轉儲(插入)到我的數據庫表中。 從下載並轉儲到數據庫后,此過程需要幾秒鍾的時間。 在這種情況下,域(記錄)的總數為34000個條目。
其次,我需要更新數據庫中每個單獨域的頁面排名,以獲取總計34000條記錄。 我有PHP API用於實時獲取頁面排名。 Godaddy下載不提供頁面等級詳細信息,因此我必須分別獲取和更新它。
現在,問題在於,要實時獲取頁面排名,然后將頁面排名更新到數據庫中,對於總共34000個域來說要花費太多時間。
我最近通過cron作業進行了一項實驗,以更新數據庫中域的頁面排名,僅花了4個小時就更新了34000個頁面中的13383個域的頁面排名。 由於必須先獲取然后更新到數據庫中。 這一切都在專用服務器上進行。
有什么方法可以加快大量域的速度嗎? 我想的唯一方法是通過多任務來完成此任務。
能否有100個任務同時獲取頁面等級並將其更新到數據庫中?
如果您需要代碼:
$sql = "SELECT domain from auctions";
$mozi_get=runQuery($sql);
while($results = mysql_fetch_array($mozi_get)){
/* PAGERANK API*/
if($results['domain']!='Featured Listings'){
//echo $results['domain']."<br />";
try
{
$url = new SEOstats("http://www.".trim($results['domain']));
$rank=$url->Google_Page_Rank();
if(!is_integer($rank)){
//$rank='0';
}
}
catch (SEOstatsException $e)
{
$rank='0';
}
try
{
$url = new SEOstats(trim("http://".$results['domain']));
$rank_non=$url->Google_Page_Rank();
if(!is_integer($rank_non)){
//$rank_non='0';
}
}
catch (SEOstatsException $e)
{
$rank_non='0';
}
$sql = "UPDATE auctions set rank='".$rank."', rank_non='".$rank_non."' WHERE domain='".$results['domain']."'";
runQuery($sql);
echo $sql."<br />";
}
}
這是我更新的pthreads代碼:
<?php
set_time_limit(0);
require_once("database.php");
include 'src/class.seostats.php';
function get_page_rank($domain) {
try {
$url = new SEOstats("http://www." . trim($domain));
$rank = $url->Google_Page_Rank();
if(!is_integer($rank)){
$rank = '0';
}
} catch (SEOstatsException $e) {
$rank = '0';
}
return $rank;
}
class Ranking extends Worker {
public function run(){}
}
class Domain extends Stackable {
public $name;
public $ranking;
public function __construct($name) {
$this->name = $name;
}
public function run() {
$this->ranking = get_page_rank($this->name);
/* now write the Domain to database or whatever */
$sql = "UPDATE auctions set rank = '" . $this->ranking . "' WHERE domain = '" . $this->name . "'";
runQuery($sql);
}
}
/* start some workers */
$workers = array();
while (@$worker++ < 8) {
$workers[$worker] = new Ranking();
$workers[$worker]->start();
}
/* select auctions and start processing */
$domains = array();
$sql = "SELECT domain from auctions"; // RETURNS 55369 RECORDS
$domain_result = runQuery($sql);
while($results = mysql_fetch_array($domain_result)) {
$domains[$results['domain']] = new Domain($results['domain']);
$workers[array_rand($workers)]->stack($domains[$results['domain']]);
}
/* shutdown all workers (forcing all processing to finish) */
foreach ($workers as $worker)
$worker->shutdown();
/* we now have ranked domains in memory and database */
var_dump($domains);
var_dump(count($domains));
?>
任何幫助將不勝感激。 謝謝
好吧,這是一個pthreads示例,它允許您對操作進行多線程...我選擇了工作程序模型,正在使用8個工作程序,您使用多少個工作程序取決於您的硬件和接收請求的服務...我從未使用過SEOstats或godaddy域拍賣,我不確定CSV字段,並且將網頁排名的獲取留給您...
<?php
define ("CSV", "https://auctions.godaddy.com/trpSearchResults.aspx?t=12&action=export");
/* I have no idea how to get the actual page rank */
function get_page_rank($domain) {
return rand(1,10);
}
class Ranking extends Worker {
public function run(){}
}
class Domain extends Stackable {
public $auction;
public $name;
public $bids;
public $traffic;
public $valuation;
public $price;
public $ending;
public $type;
public $ranking;
public function __construct($csv) {
$this->auction = $csv[0];
$this->name = $csv[1];
$this->traffic = $csv[2];
$this->bids = $csv[3];
$this->price = $csv[5];
$this->valuation = $csv[4];
$this->ending = $csv[6];
$this->type = $csv[7];
}
public function run() {
/* we convert the time to a stamp here to keep the main thread moving */
$this->ending = strtotime(
$this->ending);
$this->ranking = get_page_rank($this->name);
/* now write the Domain to database or whatever */
}
}
/* start some workers */
$workers = array();
while (@$worker++ < 8) {
$workers[$worker] = new Ranking();
$workers[$worker]->start();
}
/* open the CSV and start processing */
$handle = fopen(CSV, "r");
$domains = array();
while (($line = fgetcsv($handle))) {
$domains[$line[0]] = new Domain($line);
$workers[array_rand($workers)]->stack(
$domains[$line[0]]);
}
/* cleanup handle to csv */
fclose($handle);
/* shutdown all workers (forcing all processing to finish) */
foreach ($workers as $worker)
$worker->shutdown();
/* we now have ranked domains in memory and database */
var_dump($domains);
var_dump(count($domains));
?>
問題:
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.