如何通过命令在数据库中保存抓取的数据

Question

我在命令中有这个，但没有保存数据库。

        $client = new Client(HttpClient::create(['timeout' => 60]));
        $crawler = $client->request('GET', 'https://www.bbc.com/news/');

        $model = new Scraper();
        $crawler->filter('.gs-c-promo-heading')->each(function ($node) {
            $model->title = $node->text();         
        });
        $crawler->filter('.gs-c-promo-summary')->each(function ($node) {
            $model->text = $node->text();
        });
        $crawler->filter('.gs-c-timestamp')->each(function ($node) {
            $model->time = $node->text();
        });
        $crawler->filter('.gs-c-section-link')->each(function ($node) {
            $model->country = $node->text();
        }); 
        $model->save();       
        $this->info('done success');

给出这种类型的错误

$ php artisan scraper:start

   ErrorException

  Creating default object from empty value

  at E:\wamp64\www\Laravel7Projects\system\system\app\Console\Commands\ScrapCommand.php:49
    45|         $crawler = $client->request('GET', 'https://www.bbc.com/news/');
    46|
    47|         $crawler->filter('.gs-c-promo-heading')->each(function ($node) {
    48|            
  > 49|             $model->title = $node->text();
   
    51|         });
    52|         $crawler->filter('.gs-c-promo-summary')->each(function ($node) {
    53|             

  1   E:\wamp64\www\Laravel7Projects\system\system\app\Console\Commands\ScrapCommand.php:49
      Illuminate\Foundation\Bootstrap\HandleExceptions::handleError("Creating default object from empty value", "E:\wamp64\www\Laravel
7Projects\system\system\app\Console\Commands\ScrapCommand.php", [Object(Symfony\Component\DomCrawler\Crawler), Object(stdClass)])

  2   E:\wamp64\www\Laravel7Projects\system\system\vendor\symfony\dom-crawler\Crawler.php:352
      App\Console\Commands\ScrapCommand::App\Console\Commands\{closure}(Object(Symfony\Component\DomCrawler\Crawler))

但是使用这种方法是不需要的，因为我想将所有属性保存在一个新类中

$crawler->filter('.gs-c-promo-heading')->each(function ($node) {
            $model = new Scraper();
            $model->title = $node->text(); 
            $model->save();        
        });

我想保存数据库中的每个属性但出现错误

使用此逻辑仅保存第一行，因为我想将所有循环保存在数据库中

$client = new Client(HttpClient::create(['timeout' => 60]));
        $crawler = $client->request('GET', 'https://www.bbc.com/news/');

        $model = new Scraper();
        $crawler->filter('.gs-c-promo-heading')->each(function ($node) use ($model) {
            
            $model->title = $node->text(); 
                  
        });
$model->save();

编辑：只有标题保存在数据库中，而其他人不使用此方法保存，因为我们只在 foreach 中循环标题并且地图不起作用，而每个都在工作中，您可以检查下面的代码

$titles = $crawler->filter('.gs-c-promo-heading')->each(function($node) {
            return $node->text();
        });
        $texts = $crawler->filter('.gs-c-promo-summary')->each(function($node) {
            return $node->text();
        });
        $times = $crawler->filter('.gs-c-timestamp')->each(function($node) {
            return $node->text();
        });
        $countries = $crawler->filter('.gs-c-section-link')->each(function($node) {
            return $node->text();
        });

        $dataArray = [];
        foreach ($titles as $key => $item) {
            $newModelData = [
                'title' => $titles[$key],
                'text' => $texts[$key],
                'time' => $times[$key],
                'country' => $countries[$key]
            ];
            $dataArray[] = $newModelData;
        }
        Scraper::insert($dataArray);

Answer 1

当您使用each()您无权访问$model变量（在范围之外）。 您应该尝试use()在匿名函数的局部范围内引入$model变量

    $crawler->filter('.gs-c-promo-heading')->each(function ($node) use($model) {
        $model->title = $node->text();
    });

编辑：如果您想同时保存多个项目，您可以选择必要的值，然后根据您的模型属性创建数据数组。

    $titles = $crawler->filter('.gs-c-promo-heading')->map(function($node) {
        return $node->text();
    });
    $texts = $crawler->filter('.gs-c-promo-heading')->map(function($node) {
        return $node->text();
    });
// same for times, countries
    $dataArray = [];
    foreach ($titles as $key => $item) {
        $newModelData = [
            'title' => $titles[$key],
            'text' => $texts[$key],
            'time' => $times[$key],
            'country' => $countries[$key]
        ];
        $dataArray[] = $newModelData;
    }
    Model::createMany($dataArray);