Laravel 504網關超時

Question

我在Laravel平台中有一個腳本，該腳本可以處理成千上萬條記錄，並且超時 。 我想避免在服務器上更新設置，例如php.ini文件。

有沒有辦法將所有記錄分成小塊，並在進一步處理之前使用它們，直到所有記錄都將被使用。

我有成千上萬的記錄正在從另一台服務器獲取價格 。

所以，

1-10條記錄效果很好。

1-100條記錄效果很好。

超過500條記錄腳本卡住，並給出如下錯誤。

405 Nginx服務器不允許

我的路線就像：-example.com/updateAllKeywords

下面的代碼是獲取關鍵字以從eBay API獲取關鍵字的價格

        $keywordCollection = Keywords::all();

        Keywords::chunk(100, function ($keywordCollection) use($request,$keywordCollection,$defaultStartDate,$ignoredItems,$response) {
            foreach ($keywordCollection->toArray() as $keyword_result) {
                $response = $this->apirequest($request, $keyword_result['id'], $keyword_result, $defaultStartDate, $ignoredItems);
            }
        });

這是api請求功能：

 /**
 * Call Finding API for Active List items 
 *
 * @return API Response
 */
public function apirequest(Request $request, $id, $keyword_result, $defaultStartDate, $ignoredItems) {

    try {

        /* Request */
        $request = new Types\FindItemsAdvancedRequest();

        /* 1) default 100 items being displayed without pagination */
        $request->paginationInput = new Types\PaginationInput();
        $request->paginationInput->entriesPerPage = Config::get('constants.paginationoptions.entries_per_page');

        /* 2) ItemFilter for three conditions NEW ,NEWOTHER,USED */
        $itemFilterForConditions = new Types\ItemFilter();
        $itemFilterForConditions->name = 'Condition';
        $itemFilterForConditions->value = ['1000', '1500', '3000'];

        /* 3) ItemFilter for hide duplicate items */
        $itemFilterduplicate = new Types\ItemFilter();
        $itemFilterduplicate->name = 'HideDuplicateItems';
        $itemFilterduplicate->value[] = 'true';

        /* 4) pull items before given date / days */
        $itemFilterStartDateFrom = new Types\ItemFilter();
        $itemFilterStartDateFrom->name = 'StartTimeFrom';
        $itemFilterStartDateFrom->value[] = date('Y-m-d\TH:i:s\Z', mktime(0, 0, 0, date('m'), date('d') - $defaultStartDate['defaultStartDate'], date('y')));

        /* Exclude Keywords/MPN by set Priced Percentage */
        if (!empty($keyword_result['price']) && ($defaultStartDate['exclusionPercentage'] > 0)) {
            $percentageAmount = $keyword_result['price'] * ($defaultStartDate['exclusionPercentage'] / 100);
            $percentageAmount = floor($percentageAmount).substr($percentageAmount-floor($percentageAmount),1,2+1);
            $itemFilterPriced = new Types\ItemFilter();
            $itemFilterPriced->name = 'MinPrice';
            $itemFilterPriced->value[] = $percentageAmount;
        }

        $itemFilter = (isset($itemFilterPriced)) ? [$itemFilterForConditions, $itemFilterduplicate, $itemFilterStartDateFrom, $itemFilterPriced] : [$itemFilterForConditions, $itemFilterduplicate, $itemFilterStartDateFrom];

        $request->itemFilter = $itemFilter;

        /* 5) Request KEYWORD/MPN to filter Items Active Lists */
        $searchKeyword = (!empty($keyword_result['exclusions'])) ? $keyword_result['keyword'] . " -" . "(" . trim($keyword_result['exclusions']) . ")" : $keyword_result['keyword'];
        $request->keywords = $searchKeyword;

        /* 6) Request For sorting Items by Lowest To Highest Itemprice Price + Shipping Price */
        $request->sortOrder = 'PricePlusShippingLowest';

        /* 7) Request For Keyword Search Including the Item Descriptions,Title and Subtitle */
        $request->descriptionSearch = ($defaultStartDate['advancedExclusions'] == 1) ? true : false;

        $service = $this->index();

        /* Response returned by $request */
        $response = $this->apiresponse($request, $service, $id, $keyword_result['keyword'], $ignoredItems);

        return $response;

    } catch (\Exception $e) {
        echo json_encode(array('ack' => 'Failure', 'msg' => $e->getMessage()));
        exit;
    }
}


public function apiresponse($request, $service, $id, $keyword, $ignoredItems) {

            try {

                $response = $service->findItemsAdvanced($request);

                /* fetch all ignored Items */

                $newItems = array();
                $newOtherItems = array();
                $usedItems = array();
                $errors = "";

                if ($response->ack == "Success") {
                    $results = $response->searchResult->item;

                    ob_start();

                    foreach ($results as $item) {

                        if ($item->listingInfo->buyItNowAvailable == '1') {
                            $itemPrice = $item->listingInfo->buyItNowPrice->value;
                        } elseif (($item->listingInfo->listingType == 'Auction') && ($item->sellingStatus->currentPrice->value == 0)) {
                            $itemResponse = $this->getItemDetails($item->itemId);
                            $itemPrice = isset($itemResponse) ? $itemResponse : 0;
                        } else {
                            $itemPrice = $item->sellingStatus->currentPrice->value;
                        }

                        /* check for ignored items */
                        if (!in_array($item->itemId, $ignoredItems)) {

                            /* check for NEW Condition (conditionId = 1000) */
                            if ($item->condition->conditionId == '1000') {
                                $newItems[] = array(
                                    'keywordId' => $id,
                                    'conditionId' => (string) $item->condition->conditionId,
                                    'itemId' => $item->itemId,
                                    'title' => $item->title,
                                    'price' => $itemPrice,
                                    'itemURL' => $item->viewItemURL,
                                    'galleryURL' => isset($item->galleryURL) ? $item->galleryURL : '',
                                );
                            }
                            /* check for NEW OTHER Condition (conditionId = 1500) */ elseif ($item->condition->conditionId == '1500') {
                                $newOtherItems[] = array(
                                    'keywordId' => $id,
                                    'conditionId' => (string) $item->condition->conditionId,
                                    'itemId' => $item->itemId,
                                    'title' => $item->title,
                                    'price' => $itemPrice,
                                    'itemURL' => $item->viewItemURL,
                                    'galleryURL' => isset($item->galleryURL) ? $item->galleryURL : '',
                                );
                            }
                            /* check for USED Condition (conditionId = 3000) */ elseif ($item->condition->conditionId == '3000') {
                                $usedItems[] = array(
                                    'keywordId' => $id,
                                    'conditionId' => (string) $item->condition->conditionId,
                                    'itemId' => $item->itemId,
                                    'title' => $item->title,
                                    'price' => $itemPrice,
                                    'itemURL' => $item->viewItemURL,
                                    'galleryURL' => isset($item->galleryURL) ? $item->galleryURL : '',
                                );
                            }
                        }

                        ob_flush();
                    }

                    $newItems = array_slice($newItems, 0, 20);
                    $newOtherItems = array_slice($newOtherItems, 0, 20);
                    $usedItems = array_slice($usedItems, 0, 20);

                    $insertItems = array_merge($newItems, $newOtherItems, $usedItems);

                    /* clean up items */
                    Items::where('keywordId', $id)->delete();

                    /* save pulled items */
                    $items = Items::insert($insertItems);
                } elseif ($response->ack == "Failure") {
                    foreach ($response->errorMessage->error as $error) {
                        $errors .= "%s: %s\n\n" . $error->severity === Enums\ErrorSeverity::C_ERROR ? '<b>Error</b>&nbsp;' : '<b>Warning</b>&nbsp;' . $error->message;
                    }
                }

                /* save API log fot the keyword */
                $this->setapilog($request, $response, $keyword);

                if (!empty($errors)) {
                    echo json_encode(array('ack' => 'Failure', 'msg' => $errors));
                    exit;
                } else {
                    return true;
                }
            } catch (\Exception $e) {
                echo json_encode(array('ack' => 'Failure', 'msg' => $e->getMessage()));
                exit;
            }
        }

任何有關此的信息將不勝感激。 謝謝！

Answer 1

@金賈爾·傑斯瓦

問題：

您正在使用數據庫查詢進入foreach循環。 您正在將apirequest（）方法調用到foreach循環中。

當您的$ keywords數組具有500條記錄時。

你在打電話

    $keyword_result => 500 times. 

    $defaultStartDate = 500 times

總的來說，您在一個http請求中調用了1000個數據庫查詢。

解：

在apirequest（）函數中調用$ defaultStartDate完全沒有用。 而是在外部調用它並將值作為參數傳遞。 因此它將調用一次。

$ keyword_result可以稱為更好的方法。

$defaultStartDate = Settings::select('defaultStartDate', 
'exclusionPercentage', 'advancedExclusions')->where('currentEnv', 1)->get()-
>first()->toArray();

$keywordValues = array_keys($keywords);

$keywordCollection = Keywords::whereIn('id', $keywordValues)-
>select('keyword', 'exclusions', 'price')->first()->toArray();

foreach ($keywordCollection as $keyword_result) {

$response = $this->apirequest($request, $keyword_result->id, 
$keyword_result, $defaultStartDate);

}

/**
  * Call Finding API for Active List items
  *
  * @return API Response
  */
function apirequest(Request $request, $id, $keyword_result, $defaultStartDate)
{
 // your code here.
}

這只是您可以使用的代碼結構。

Answer 2

創建一個有工作的Laravel Queues將函數放在handle方法中。

$response = $this->apiRequest(...);

從您的控制器說出您的工作

foreach ($keywordCollection as $key => $keywordResult) {
   ProcessJob::dispatch($keywordResult)->delay($key * 2); 
}

這樣，您就實現了指數補償，從而隨着數組的增加，您增加了等待下一個api請求的時間。